Sub-display designation for remote content source device

ABSTRACT

Disclosed in some examples are display systems, methods, devices, and machine-readable mediums which provide for a gesture-based method for specifying a region of a display in which to show shared content. Also disclosed are pairing methods for associating the region of the display with a content source device, methods for sharing content within the region, and methods for providing input from the region back to the content source device.

BACKGROUND

Large displays, such as those that cover entire walls, billboards, andsides of buildings are useful for providing content during meetings,tradeshows, speeches, advertising, and other events. These displays maybe made of a single display or may be composed of multiple smallerdisplays connected through hardware and software to form one largedisplay. In some examples, the displays may have input capability, suchas by detecting touch input or gestures using a touch screen, or camerasthat detect gestures and/or touch events.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, which are not necessarily drawn to scale, like numeralsmay describe similar components in different views. Like numerals havingdifferent letter suffixes may represent different instances of similarcomponents. The drawings illustrate generally, by way of example, butnot by way of limitation, various embodiments discussed in the presentdocument.

FIGS. 1-3 illustrates display systems according to some examples of thepresent disclosure.

FIG. 4 illustrates designating a sub-display of a display using agesture according to some examples of the present disclosure.

FIG. 5 illustrates gesture-based designation of sub-displays accordingto some examples of the present disclosure.

FIG. 6 illustrates a designation and pairing process according to someexamples of the present disclosure.

FIG. 7 illustrates a designation and pairing process using biometricsaccording to some examples of the present disclosure.

FIG. 8 illustrates a designation and pairing process for a remotecontent source device according to some examples of the presentdisclosure.

FIG. 9 illustrates a data flow according to some examples of the presentdisclosure of a remote content source device that is not physicallyproximate to the display sharing content on the display through acommunication service intermediary according to some examples of thepresent disclosure.

FIG. 10 illustrates a display with two designated sub-displays, one forinput, according to some examples of the present disclosure.

FIG. 11 illustrates an example of a sub-display with a notification areaaccording to some examples of the present disclosure.

FIG. 12 shows an example of a graphical user interface (GUI) of acommunication application according to some examples of the presentdisclosure.

FIG. 13 illustrates a display with multiple sub-displays correspondingto multiple communication modalities of the communication applicationaccording to some examples of the present disclosure.

FIG. 14 shows an illustration of a suggested additional sub-displayaccording to some examples of the present disclosure.

FIG. 15 illustrates a data flow of a suggested sub-display according tosome examples of the present disclosure.

FIG. 16 shows an illustration of a suggested display size changeaccording to some examples of the present disclosure.

FIG. 17 illustrates a logical diagram of a display control systemaccording to some examples of the present disclosure.

FIG. 18 illustrates a method flow of a service advertisement accordingto some examples of the present disclosure.

FIG. 19 illustrates a method flow for handling the sub-display creationnotification according to some examples of the present disclosure.

FIG. 20 illustrates a flowchart of a method for handling content from acontent source device at a display control system according to someexamples of the present disclosure.

FIG. 21 illustrates a flowchart of a pairing operation according to someexamples of the present disclosure.

FIG. 22 illustrates a flowchart of a method of routing input to acontent source device according to some examples of the presentdisclosure.

FIG. 23 illustrates a flowchart of a method of providing a notificationto a sub-display according to some examples of the present disclosure.

FIG. 24 illustrates a flowchart of a method of a notification componenthandling an input to a notification according to some examples of thepresent disclosure.

FIG. 25 illustrates a logical diagram of a display according to someexamples of the present disclosure.

FIG. 26 illustrates a flowchart of a method of registering a displaywith a controller according to some examples of the present disclosure.

FIG. 27 illustrates a flowchart of a method of a sub-display creationperformed by a sub-display creation component according to some examplesof the present disclosure.

FIG. 28 illustrates a flowchart of a method of a content displaycomponent causing content to be displayed according to some examples ofthe present disclosure.

FIG. 29 illustrates a flowchart of a method of handling input directedto a sub-display according to some examples of the present disclosure.

FIG. 30 illustrates a flowchart of a method of a notification handlercomponent handling a notification message from a content source deviceor a display control system according to some examples of the presentdisclosure.

FIG. 31 illustrates a flowchart of a method of a notification handlercomponent handling a gesture directed to the notification area,according to some examples of the present disclosure.

FIG. 32 illustrates a diagram of a content source device 3210 accordingto some examples of the present disclosure.

FIG. 33 illustrates a flowchart of a method of sharing content on asub-display of a remote display according to some examples of thepresent disclosure.

FIG. 34 illustrates a flowchart of a method of processing input from asub-display according to some examples of the present disclosure.

FIG. 35 illustrates a flowchart of a method of presenting a notificationto a sub-display according to some examples of the present disclosure.

FIG. 36 illustrates a flowchart of a method of handling an input such asa gesture directed to a notification displayed on a sub-display.

FIG. 37 illustrates a flowchart of a method of designating a sub-displayaccording to some examples of the present disclosure.

FIG. 38 illustrates a flowchart of a method of pairing according to someexamples of the present disclosure.

FIG. 39 illustrates a flowchart of a method of an input sub-displayaccording to some examples of the present disclosure.

FIG. 40 illustrates a flowchart of a method of providing notificationsaccording to some examples of the present disclosure.

FIG. 41 illustrates a flowchart of a method of designating a sub-displayarea for a remote content source device that is not in physicalproximity to the display according to some examples of the presentdisclosure.

FIG. 42 is a block diagram illustrating an example of a machine uponwhich one or more embodiments may be implemented.

FIG. 43 illustrates a content splitting flow according to some examplesof the present disclosure.

DETAILED DESCRIPTION

To specify content for presentation on a display a graphical userinterface (GUI) on a controller device (e.g., a computing device) istypically provided. Users select the content, placement, and layoutoptions for the display through the GUI. Content is sent from thecontroller device to the display and is displayed according to theselected layout and placement. Organizations deploying these systemsoften have an administrator or some other user that is familiar with,and has access to, the GUI of the controller device. While these systemsare often powerful and provide flexibility in layout options, they arenot practical for a shared space like a meeting room where manydifferent people are expected to interact with and use the display atvarious different times. Hiring an administrator to set up a displaythat will run in a particular configuration for a long time (e.g., likein a digital billboard or trade show) may be cost effective, having anadministrator constantly on call to customize the system for users in ashared space (such as a meeting room) is not. As a result, manyorganizations make the display control system GUI publicly accessiblefor users to customize the layout for their individual purposes. Thisputs the burden on users that may not be familiar with the GUI of thedisplay control system which may produce sub-optimal experiences withthe display.

In order to solve this problem, other display systems have simplifiedthe user interfaces to specify the content, placement, and layoutoptions. This simplification is easier for users because it provides amore limited interface for users to control but comes at the cost ofless flexibility. For example, large televisions may allow for thedisplay of multiple different content sources on a single screen, butthese systems typically provide only a limited number of preset layoutoptions that are accessible via a remote-control interface.Additionally, these systems lack sophisticated input handlingmechanisms.

Current large display systems thus lack flexible and easy to use contentselection, layout, and placement options to allow users to quickly andeasily place content on the display where they want it. As noted,current display systems typically feature proprietary interfaces withstatic layout options that may not be available to end users, or theymay feature more limited interfaces that do not allow appropriateflexibility in designating content for display. As used herein, contentmay refer to one or more of documents, document data, video, audio,screen sharing data, application sharing data (e.g., video frames froman application), and/or the like.

Disclosed in some examples are display systems, methods, devices, andmachine-readable mediums which provide for a gesture-based method forspecifying a sub-display e.g., a region of a display in which to showshared content. Also disclosed are pairing methods for linking (e.g.,associating) the sub-display with a content source device, methods forsharing content within the region, and methods for providing input fromthe region back to the content source device.

A sub-display is a defined region of the display in which content from aparticular content source device is displayed. The sub-display may beindependent of any other sub-display in that a size, shape, position,settings, and content source device of each sub-display may bedifferent. A user may perform a gesture over, on, or directed to, adisplay area to designate that area of the display as a sub-display forsharing content. The gesture may be a touch-gesture, or may be an airgesture that may be detected by a camera. The input designating thesub-display may specify the coordinates and shape of the sub-display onthe display. For example, a user may touch the display and drag theirfinger over the display to draw an outline of the sub-display at theposition the user touched. As used herein, a sub-display is a portion ofa larger display that may be paired (i.e., linked) with a content sourcedevice to display content selected by the content source device. Eachsub-display may display different content from one or more othersub-displays on the same display. The content source device paired withthe sub-display may communicate content to the display through a directconnection or through a network connection. Input from the sub-displaymay be directed to the content source device that is paired with thesub-display.

Each sub-display may be graphically distinguished from a surroundingdisplay area (which may include other sub-displays) by a graphicaloutline, a border, frame, window, or the like. Example inputs that causedesignation of a sub-display include a user tracing their finger overthe display at the location and in the shape of the desired sub-display.The shape may be a square, a circle, triangle, polygon, or the like.Other example gestures may include a drag operation where the point atwhich the user's finger contacts the display is a first corner of thesub-display and the user drags their finger and releases their finger ata second corner of the sub-display—the second corner being diagonal tothe first corner. The display is configured to detect gestures throughthe use of touch screens, cameras, or other input devices.

To display content in the sub-display, after (or in some examplesbefore) designating the sub-display the user may specify a contentsource device that provides the content that is displayed in thesub-display by pairing the sub-display with the content source deviceusing a PIN, biometric, QR code, or other method. Once paired, thecontent source device may send content to the display for sharing on thesub-display. Both the designation of the sub-display and the linking ofthe sub-display to one or more applications is accomplished without theneed for utilizing a separate user interface of a controller device.

The disclosed display systems may allow one or more applications on asingle content source device to cast the same or different content tomultiple sub-displays on the display simultaneously. Similarly, multiplecontent source devices may simultaneously cast the same or differentcontent to multiple different sub-displays of a same displaysimultaneously.

The disclosed display systems may also be configured to detect userinput, including touch inputs, gesture inputs, and/or the like directedto or within one or more of these sub-displays. Inputs may be detectedsimultaneously on or in multiple sub-displays. These inputs may be sentback to the application linked to the sub-display on the content sourcedevice and processed by an application on the content source device(such as the application that provided the content shared on thatsub-display). In some examples, the input may be processed as if theywere inputs produced by an input device that is directly coupled to thecontent source device. In some examples, a sub-display may be designatedas an input-only sub-display and may be separate from any sub-displaythat displays content. The input sub-display may display any enteredinput or may hide the entered input in order to provide a private areafor input.

In some examples, input into a sub-display may be provided by a thirddevice. For example, a content source device (e.g., a laptop) mayprovide content that is displayed within the sub-display. A contentsource device may be any computing device that is capable of sharingcontent in a sub-display. A third device, such as a tablet, may allowusers to provide input into the sub-display. In order to accomplishthis, the separate input device (e.g., the tablet) may pair (using thesame methods as disclosed herein for the content source device) with thesub-display. The pairing may be an input only pairing, or an inputand/output pairing. Input from the tablet may be sent by the displaycontrol system (or in some examples, the display control system mayfacilitate a peer-to-peer (P2P) connection between the input device andthe content source device by providing a P2P address of the contentsource device to the input device) to the content source device where itis processed and reflected in any change in the content sent to thedisplay control system.

More generally, it is contemplated that multiple input and/or outputdevices may be paired to a single sub-display. In the case of multipleoutput and input devices, user configurable rules on the display controlsystem may govern how conflicting input and/or output is handled. Forexample, different content sources may be displayed side-by-side in thesub-display. While multiple input and/or output devices may be paired toa single sub-display, in other examples a sub-display may accept contentand send input to a single content source device.

The sub-displays may also display one or more notifications in anotification area. The notifications may be sent by the paired contentsharing device. Notifications may appear within the sub-display, or in anotification area visually attached to, or adjacent the sub-display.Users may interact with notifications, such as to dismiss thenotification, reply to the notification, obtain more information aboutthe event tied to the notification, or the like.

In addition, while the aforementioned designation and pairing processesdescribe designation and pairing for a user that is in a same room tothe display, in some examples, a remote user may utilize an intermediaryserver (such as a communication server) and a local user to sharecontent with a sub-display of a display that is not in a same locale asthe remote user.

By utilizing gesture-based designation of sub-displays, users mayquickly and easily designate content for a portion of the display.Because the gesture is drawn relative to the display itself, rather thanon a controller's user interface, user's may visually layout the displayto more naturally achieve a desired layout and content selection. Thisavoids problems in traditional display control system user interfaces inwhich it may be hard to visualize each layout choice when applied to thelarger display (rather than the user interface which may be on a smallerscreen). This also avoids problems that plague simpler user interfacesthat may offer less flexible layout options as in the disclosed systemthe size and shape of each sub-display is “drawn” by the user usinggestures and is thus completely within the user's control.

Thus, the present disclosure discloses improved user interfaces forcomputing, such as for large displays. The improved display interfacesallow users to more quickly share content of a content source device ina desired place and layout on the display. The disclosed methods arefaster and easier than conventional solutions while still preservingflexibility. Rather than paging through multiple screens of options indisplay control system interfaces or dealing with more limited optionsfor more user-friendly interfaces, in the disclosed system, the useronly needs to draw the sub-display on the larger display, select asharing option in their content sharing device, and enter pairinginformation. The present disclosure also eliminates the need forseparate display control system GUI interfaces, thus simplifying thedesign of the display control system without sacrificing the flexibilityoffered by those GUIs. The present disclosure thus solves the technicalproblem of layout, organization, and selection of content on a displayby the technical solution of utilizing gesture-based input, pairing, andcontent sharing. The present disclosure improves the functioning of thecomputer system by providing improved user interfaces by distillingcomplex layout and setup procedures into an intuitive and easy to useinterface and at the same time reducing processing load of the displaycontrol system by the elimination of the GUIs—all while maintainingflexibility and features of more complex systems.

FIG. 1 illustrates a display system 100 according to some examples ofthe present disclosure. A display 120 is communicatively coupled to adisplay control system 110. The display 120 and the display controlsystem 110 may communicate through wired or wireless means and may bedirectly coupled or coupled through a network, such as a packet-basednetwork. In some examples, the display control system 110 may controlone or more displays such as display 120. In some examples, the display120 registers with the display control system 110 to be controlled bythe display.

For example, the display control system 110 may be any computing device(whether separate from the display 120 or integrated with the display120) that manages sub-displays on a display. The display 120 may send abroadcast message with data in the message indicating that it is adisplay looking for a controller. The message may be sent to aparticular defined port and to a particular defined broadcast address.The display 120 may then listen for these messages and reply to thedisplay control system 110 at the address from which the broadcastmessage is sent. Once the display 120 and display control system 110find each other and determine a network or other address, the displaycontrol system 110 and the display 120 may then negotiate one or moreparameters used by the display control system 110 to control the display120. Example parameters may include network addresses to send content,network addresses where input is sent from the display to the displaycontrol system, how pairing is to be performed, parameters of thedisplay such as the display size and capabilities, and the like. Thedisplay control system 110 may then register the display 120 in adatabase of controlled displays. For example, the display control system110 may create a database record with the parameters, addresses, andcapabilities of the display. While the display control system 110broadcasts a message in the above example, in other examples, thedisplay 120 broadcasts a message looking for a display control system.

Display 120 and/or display control system 110 may recognize one or moreinputs directed to the display 120 for creating one or moresub-displays. For example, the display 120 may be a touch screendisplay. In other examples, the display 120 may have a camera pointed atthe display or pointed at a user of the display. The camera may becommunicatively coupled to the display 120 or the display control system110 that may detect gestures. A user touching the display outside of analready created sub-display along with a dragging gesture may berecognized as a command to create a new sub-display. The geometry of thegesture may determine the geometry of the sub display. For example, theuser may trace an outline of the desired sub-display. In some examples,this geometry may be corrected by the system (e.g., a user's attemptedcircle with uneven lines and varying radius may be corrected to be acircle with a constant radius).

In some examples, visual feedback may be provided to the user as theycreate the sub-display showing the boundaries of the sub-display as itis created. For example, an outline may be shown of the currentlycreated sub-display as the user moves their finger over the display tocreate the sub-display. In other examples, instead of an outline, asub-display may be created as a box with opposite corners being a firstand last touch points in a drag operation. As the user drags from onecorner to the opposite corner, the display may display an expandingwindow or box.

In examples in which an input device communicatively coupled to thedisplay control system 110 identifies the gesture to designate asub-display (e.g., a camera coupled to the display control system 110that may take video of the display to detect input), the display controlsystem 110 may send a message indicating an input designating a newsub-display and the coordinates of the input to the display 120 fordisplaying the visual feedback (e.g., for display of the outline orother visual designation of the boundaries of the sub-display during thegesture).

Once designation of the size and shape of a sub-display is complete, thedisplay 120 and display control system 110 may create a sub-display andpair the sub-display with a content source device, such as contentsource device 130.

In examples in which the display 120 detects the gesture to designatethe sub-display, the display 120 may record the coordinates of thesub-display, as entered by the user, and may communicate the coordinatesto the display control system 110. The coordinates may be sampled pointsalong the user's trace, or may be coordinates of a box designated by theuser by determining a first corner where the user first presses thescreen and a second corner where the user lifts their finger off thescreen. In examples in which an input device coupled to the displaycontrol system 110 recognizes the gesture to designate the sub-display,the display control system 110 may send the coordinates to the display120. Once a sub-display is designated additional user inputs may move,resize, change the shape of (e.g., from a first polygon to a differentpolygon or to a circle), and close a sub-display.

When a sub-display is designated, the display 120 and/or display controlsystem 110 may create data structures that store information about thesub-display. For example, the coordinates of the sub-display, a uniqueidentifier of the sub-display (e.g., a PIN number of the sub-display),information on the display it was created on, and the like. The datastructure may have fields that may be populated later with informationabout the content source device once a content source device is pairedwith the sub-display. For example, an address of the content sourcedevice, an application identifier of the content source device, and thelike that are used to accept content from the content source device andmatch that content to a sub-display and also to route input from thesub-display and route it to the correct application on the contentsource device.

To pair the sub-display on the display 120 with the content sourcedevice 130, one or more methods of pairing may be used. For example, thecontent source device 130 (e.g., a computing device) may discover thedisplay control system 110. The display control system 110 may broadcastadvertisements for a display service and the content source device 130may listen for these advertisements. The broadcasts may indicate anaddress of the display control system 110. The content source device 130may save this address for future streaming of content and pairing to thedisplay control system 110.

When the sub-display is created, the display 120 may display a PIN, QRcode, password, or other unique identifier of the sub-display. This maybe displayed in a graphical user interface that is in the sub-display,proximate to the sub-display, visually touching but not overlapping thesub-display, or the like. For example, in a box that is below andvisually connected but not overlapping the sub-display. For example,FIG. 6 shows a PIN in a pairing tab 630 below the sub-display. This pinmay be created by either the display 120 or the display control system110.

This pin may then be entered by the user into an application on thecontent source device 130. For example, based upon discovery of thedisplay service, the content source device may provide a user an optionto share the content of a particular application to a display, such asdisplay 120. For example, through a context or sharing menu. Theapplication may then send the entered PIN to the discovered displaycontrol system 110. The display control system 110 may then match theapplication of the content source device 130 with the sub-display on thedisplay 120. The content source device 130 may then begin sharingcontent to the sub-display by sending the content to the display controlsystem 110 and may receive input from the display 120 via the displaycontrol system 110.

Content source device 130 may communicate with the display controlsystem 110 and/or the display 120 through various wired and wirelessmeans, including through a network, such as a packet-based network.While a display control system 110 is shown, one of ordinary skill inthe art will appreciate that the functions of the display control system110 may be implemented in one or more other devices as well as thedisplay 120.

Display 120 may be a liquid crystal display (LCD), a light emittingdiode (LED) display, an organic LED (OLED) display, a plasma display, orthe like. The display 120 may include a touchscreen. The display 120 ordisplay control system 110 may also be coupled to a camera that detectsgestures directed to the display. Display control system 110 may be acomputing device, such as a server computing device, a laptop, adesktop, a tablet, a smartphone, a dedicated hardware controller, or thelike. Display control system 110 may be attached to, or a same device asthe display 120 and communicatively coupled through a local connectionto the display 120 or may be a different device communicatively coupledthrough a network to display 120. Content source device 130 may be acomputing device, such as a laptop, desktop, tablet, smartphone, smartwatch, or the like.

FIG. 2 illustrates a display system 200 according to some examples ofthe present disclosure. In the illustration of FIG. 2, once the contentsource device 130 is paired with the sub-display, the content sourcedevice 130 may send the content directly to the display 120 rather thansending it to the display control system 110 as in FIG. 1. This may beaccomplished by the display control system 110 providing the address ofthe display 120 to the content source device 130 during pairing.

FIG. 3 illustrates a display system 300 according to some examples ofthe present disclosure. In FIG. 3 the display control system is part ofthe display 120. The display 120 in FIG. 3 directly communicates withthe content source device 130 for discovery, pairing, and contentsharing.

Sub-Display Designation

FIG. 4 illustrates designating a sub-display of display 120 using agesture according to some examples of the present disclosure. Thegesture of FIG. 4 may be a drag gesture when the user first touches thedisplay 120 at first point 410-A and drags their finger in a diagonalmotion to points 410-B, and 410-C. The designated sub-display would bethe rectangle with diagonal vertices at the first point 410-A and thepoint 410-C where the user takes their finger off the display. In someexamples, after the first touch at first point 410-A and a subsequentdrag towards point 410-B, the display 120 may provide an outline orother graphical representation of a window, such as representation 420.As the user moves their finger the representation updates to match thecurrent size and shape of the sub-display. While in FIG. 4, a rectangleis displayed, in other examples, other shapes may be designated the sameway with sizes scaled to occupy a bounding box with vertices at thebeginning and ending points of the gesture.

In FIG. 4, other sub-displays are represented as well. For example,sub-display 430 which is showing content A shared from content sourcedevice 1. The content may be from a particular application on contentsource device 1—for example, a first application from content sourcedevice 1. Sub-display 440 displays content A from content source device2, for example, from a first application from content source device 2.Sub-display 450 displays content B from content source device 1, forexample, content from a second application of the content source device1.

FIG. 5 illustrates other gesture-based designation of sub-displaysaccording to some examples of the present disclosure. For example,instead of a drag operation where the diagonal vertices are specified bythe beginning and ending points of the drag, the user may specify theoutline of the sub-display. For example, the user may trace a square,rectangle, polygon, circle, or the like and the sub-display may utilizethe path traced by the user as the coordinates of the sub-display. Thesub-display shape may be the exact shape traced by the user, that is, itmay not be updated to be a smooth shape such that imperfections in theuser's tracing of the shape may be reflected in the sub-display. Inother examples, the system may, automatically, or in response to a userinput—correct the user's trace to match a closest shape from a libraryof shapes. For example, as shown in FIG. 5, the user's trace 510 iscorrected to a square 520. A user's trace 530 is corrected to an octagon540, and the user's trace 550 is corrected to be a circle 560.

The system may achieve the correction by straightening lines betweenvertices in the user's trace. Vertices may be identified by identifyingdirection changes e.g., by computing the slope of a line betweensuccessive coordinates of the user's trace and comparing the slopeperiodically throughout the user's trace and comparing it to athreshold. In other examples, the completed shape is compared to alibrary of shapes. The system may calculate a total difference in (x, y)coordinate points between the points and/or vertices of each libraryshape and the outline that the user traced. The system may select theshape with the lowest total difference. In some examples, in order toproperly normalize the size, the shape traced by the user may be scaledto a normalized size for the comparison, or the shape in the shapelibrary may be normalized to a size corresponding to the user's trace.

In some examples, the display may visually indicate the geometriccorrection by animating a movement of the lines drawn by the user'strace to the corrected geometric shape. The sub-display may take thesize, and boundaries indicated by the corrected geometric shape.

In some examples, whether the trace is geometrically corrected or notmay be set by one or more settings. A global setting may be used that isapplied to each sub-display designation without user input during thedesignation. In some examples, a setting may be set on a per sub-displaydesignation basis. For example, the user may apply a user input eitherbefore, during, or after the designation of the sub-display to designatewhether the system should correct the shape drawn by the user. Forexample, during the sub-display designation (e.g., during the tracing ofthe shape), if the user taps the display with a different finger (whilethe user is still tracing with the other finger)—a menu may be displayedthat allows the user to activate the geometric correction. In otherexamples, a menu may allow the user to select the shape to correct thetrace to.

In order to conform the content to the selected shape, the contentsource device, the display, and/or display control system may apply oneor more transformations to the content, such as resizing the content,stretching the content, skewing the content, cropping the content (e.g.,to the shape of the sub-display), rotating the content, and the like.

Pairing Operations

To route content from a content source device to the correct sub-displayon the display, the content source device may pair (i.e., link) with thesub-display. Pairing (i.e., linking) is used herein to describe aprocess for associating the content source device with a particularsub-display to allow the routing of content from the content sourcedevice to a particular sub-display. The pairing process may happenbefore or after the designation of the coordinates of the sub-display.FIG. 6 illustrates a flow diagram of a designation and pairing processaccording to some examples of the present disclosure. In the example ofFIG. 6, the user designates the coordinates of the sub-display 610 bydesignating the corners of the sub-display. The rest of the coordinatesare calculated by the system based upon those two corners. Thesub-display may include a GUI window with controls for resizing,minimizing, maximizing, and closing the sub-display. In other examples,the sub-display may be frameless with no visible controls. One ofordinary skill in the art with the benefit of Applicant's disclosurewill appreciate that other visual configurations are contemplated.

In some examples, a pairing tab 630 may be shown adjacent to thesub-display 620 with pairing information. As shown in FIG. 6, thepairing tab 630 is connected to one edge, but not overlapping thesub-display. In some examples, the pairing tab 630 may be displayed inthe sub-display 620. In other examples, the pairing information inpairing tab 630 may be displayed in the sub-display as text. Forexample, as shown in FIG. 6, the sub-display currently displays the text“Please enter the PIN in your sharing application.” In some examples,the PIN may be displayed along with the message, or may be displayedinstead of the message or the like. The pairing information displayed inthe pairing tab 630 may be a PIN number. The PIN number may be createdby the display, the display control system, or the like. The PIN numbermay be a sub-display identifier that uniquely identifies thatsub-display to the display and/or the display control system.

The content source device may be executing an application that may allowfor casting (i.e., sharing) of content. For example, a communicationapplication, a photo application, a productivity application (e.g., aword processing application, a spreadsheet application, a drawingapplication, or the like), and the like. Example communicationapplications may be a remote meeting application, a video chatapplication, a video playback application, a photo application, and thelike. In some examples, the content source device may allow sharing ofone or more of the displays of the remote display device. For example,sharing a desktop of the remote display device.

In order to share content, the user may select a share contentoption-either in the application that has the content they wish toshare, or via a sharing utility (e.g., provided by an operating systemof the remote content display device). The share option to the displaymay be enabled in response to the content source device discovering thedisplay service (e.g., by receiving broadcast messages sent by thedisplay control system). In the example of FIG. 6, a GUI 640 of acommunication application is shown according to some examples of thepresent disclosure. In FIG. 6, the communication application isparticipating in a network-based communication session (such as anonline meeting) and the user has indicated that the user would like toshare a window. An option is displayed that allows the user to determinewhere to share the content, either with other remote participants of thenetwork-based communication session (“meeting”) or on a remote display(“remote display”). If the display is selected, a menu or window isdisplayed that allows the user to enter the PIN displayed on thedisplay.

As already described, an application on the content source device (e.g.,either an application that allows content sharing, or an operatingsystem service that provides a sharing service to other applicationsthat share content) discovers a display service advertised by a displaycontrol system. When a display service is discovered, the applicationenables sharing content to the display. When a user indicates to sharecontent, the application or service on the content source device sends acontent share request with the entered PIN to the address of the displaycontrol system discovered during the discovery process of the displayservice. If the PIN matches a displayed PIN, the display service maypair the content source device to the sub-display. The displayed PIN maybe sent by a display to the display control system. In other examples,the displayed PIN may be determined by the display service provided bythe display control system (and sent to the display for display). Forexample, the display control system and/or display may recordinformation about the content source device in a data structuredescribing the sub-display. Furthermore, content from the content sourcedevice may be routed to the sub-display and input from the sub-displayrouted to the content source device based upon the pairing. The displayservice may send a pairing confirmation to the content source device.The content source device then sends the content to either the displaycontrol system or the display itself—depending on the implementation.Content shared may be video, audio, still images, application displays,or the like. Once the display is paired and streaming begins, then at660 the content is displayed in the sub-display.

In other examples, the pairing may operate in the reverse. That is, theapplication on the content source device may specify the PIN and theuser may enter the PIN in a pairing input area (e.g., such as a pairingtab) of the display. In these examples, the PIN may be sent by thecontent source device to the display control system. This PIN is thenmatched to a PIN entered into the pairing tab of a sub-display.

Pairing

FIG. 7 illustrates a flow diagram of a designation and pairing processusing biometrics according to some examples of the present disclosure.As in FIG. 6, the user begins by designating the coordinates of asub-display 710. The displayed sub-display 720 may have a pairing tab730. Pairing tab 730 may be a biometric entry area. For example, afingerprint entry area. The display may have an integrated fingerprintreader that may allow for fingerprint entry in one or more locations ofthe display. In some examples, the display may have the ability to scanfor a fingerprint over the entire display and the pairing tab 730 may beanywhere on the display. In some examples, the pairing tab 730 may beadjacent to and connected to a border of the sub-display 720. In theseexamples, the pairing tab 730 serves to limit the processing powerrequired for scanning for fingerprints by limiting the area that needsto be scanned to the coordinates of the pairing tab 730. Rather thanscanning the entire surface of the display, the display only needs toscan for a biometric in the pairing tab 730. Additionally, the pairingtab 730 serves to link the sub-display with the scanned biometric suchthat multiple sub-displays may be created and linked simultaneously.

When the user tries to pair with a sub-display, the user enters thebiometric at the display, such as in the pairing tab 730. The biometrictemplate may be sent to the display control system who stores thebiometric template and associates it with the designated sub-display(e.g., based upon which sub-display's pairing tab scanned thebiometric). The user selects to share content with a display byselecting a share option in a content sharing application (such asthrough a menu 750 of a GUI of the content sharing application 740). Thecontent source device may send a biometric template of the same type(e.g., a fingerprint scan) that is taken by the content source device(either at the time of pairing, or during a registration or setupprocess) to the display control system. The display service thensearches for a match between the biometric template sent by the contentsource device and biometric templates entered into pairing tabs (orother locations) of unpaired sub-displays. If a match is found, thecontent source device and the sub-display may be paired and a messagemay be sent to the content source device that a match is found andcontent may begin sharing, such as shown at GUI 760.

In some examples, in order to protect the privacy of the users, thefingerprint and/or other biometric may use a non-standard template typesuch that if a malicious party were to obtain the biometric template(either stored in the application or with the display service), thetemplate would not be usable to bypass biometric authentication withother services or applications. In some examples, the biometrictemplates may be encrypted. In some examples, the biometric templatesstored at the display control system may be deleted from the controllerafter a pairing process. If no pairing process results with a particularbiometric template, that template may be deleted after a short timeperiod.

In some examples, the pairing tab 730 may be utilized for capturingbiometrics, but in other examples, other specific locations for enteringthe biometric may be utilized. For example, a retinal scanner may belocated at a particular place on the display. For examples in which abiometric scanner is in a central location, the system may restrict thecreation of sub-displays such that a single sub-display may be paired atonce. This allows for linking a particular sub-display with a particularcaptured biometric. A biometric in these examples may be obtained at thedisplay by the user presenting themselves for a scan at the display ornear the display (e.g., within a defined distance). The biometricscanner may be in a same room, adjacent to the display (e.g., mounted ona same wall surface as the display), facing the display (e.g., the usermay turn to have their back face the display to have a face or eyescan). The biometric scanner may be connected through a local interface(such as a Universal Serial Bus interface, Serial Interface, or thelike), in other examples, the biometric scanner may be connected througha network interface

In other examples, the system may allow for the creation of multiplesub-displays simultaneously with a central biometric scanner location.In these examples a method for specifying which sub-display a biometricscan is associated with may be utilized. For example, a user may enteran identification name and/or number of a sub-display into an input areaof the display setup for this purpose just prior to, or just afterscanning of the biometric.

Remote Display Sharing

While the pairing procedures described herein are utilized for users ofcontent source devices that have physical access to the display at thetime of sub-display designation and pairing, in other examples, userswith content source devices that do not have physical access at the timeof sub-display designation and pairing may share content in asub-display of the display. These content source devices are referred toherein as remote content source devices. Remote content source devicesare content source devices whose users do not have physical access tothe display to pair with a sub-display.

A communication application such as MICROSOFT TEAMS® may allow remoteusers to share content on a display in a conference room for localparticipants to view and interact with. In some examples, this may beachieved by one of the local participants receiving the shared contentthrough an network-based communication session (e.g., an online meeting)of the communication application. The local participant may designate asub-display on the display and pair the communication applicationexecuting on that local participant's computing device to thesub-display. This causes the sub-display to display whatever the localparticipant's device is displaying. Thus, if the local participant isviewing the shared content of the remote user in the communicationapplication, the sub-display will also display that content. There aredrawbacks to this approach. For example, if the local participant leavesor changes what is displayed in the paired application, this may stop orinterrupt the content sharing for other local participants who arerelying upon the content sharing. Thus, the continued display of contentof the remote user is dependent on the local participant. Furthermore,some communications applications may not be setup to allow one user'sshared content to be re-shared to another location.

In some examples, the system may pair a remote content source devicethat is not physically proximate to a sub-display of the display withoutrelying on a local computing device for continued content delivery. Bynot physically proximate, it is meant that a user of the remote contentsource device cannot see the display. FIG. 8 illustrates a flow diagramof a designation and pairing process for a remote content source deviceaccording to some examples of the present disclosure. A user in physicalproximity to the display first designates a sub-display by usinggestures 810 as previously described. The system may then show a PIN orother pairing mechanism as previously described, such as within apairing tab 830 of the sub-display 820. A local user on a localcomputing device may be executing a communication application, such asapplication 840. The communication application may have an option, suchas within a menu 850 for designating a content source device of a remoteuser for sharing to a sub-display of a local display. In FIG. 8, theoption is listed as “remote display share” and upon selecting thisoption, a second menu may be displayed that allows a user to enter a pinat a first input area 860 of a created sub-display and designate aremote user's content source device at a second input area 870. Uponselection of these options, a communication may be sent to the remoteuser's computing device.

An instance of the communication application executing on the remotecomputing device then allows the remote user to select the application,window, or other content to share with the remote display, such asthrough a menu 890 of a GUI 875. Once the remote user selects theappropriate content to share, the communication application of theremote user may stream the content to the sub-display of the display,such as shown at 880. In some examples, a communication serverassociated with the communication application may facilitate thecommunication between the various instances of the communicationapplication and with the display and/or display control system.

FIG. 9 illustrates a data flow according to some examples of the presentdisclosure of a remote content source device that is not physicallyproximate to the display sharing content on the display through acommunication service intermediary according to some examples of thepresent disclosure. The local computing device 920 (which may also serveas a content source device in different contexts) may discover, aspreviously described, a display service at operation 902 bycommunicating with the display control system 940. As part of thediscovery process, the local computing device 920 may discover anaddress of the display control system 940. In some examples, due to theremote content source device 910 not being in physical proximity to thedisplay control system 940, the remote content source device 910 may nothave a way of discovering the display control system 940. For example,the broadcast messages of the display control system 940 may not reachthe remote content source device 910 because they may only be broadcastover a particular sub-net or other network area that is not accessibleto the remote content source device 910. In other examples, thebroadcast messages may be wireless and the remote content source device910 may be out of wireless range.

Next, at operation 904, a user of the local computing device 920 maydesignate a sub-display on the display using the methods previouslydescribed (e.g., by using gestures to draw an outline of thesub-display). At operation 906, the display then communicates with thedisplay control system 940 to establish the pairing information, and arecord of the sub-display. For example, by creating a data structuresuch as shown at 950 that has the coordinates of the sub-display, a PINnumber (which uniquely identifies this sub-display on the display), acontent source device info (which might be empty prior to pairing), andthe display identifier (which uniquely identifies the display). Aspreviously noted in some examples, the PIN is assigned by the displaycontrol system 940, but in other examples is created by the display. Insome examples, the PIN is a unique identifier for the sub-display acrossthe display 930, but in other examples, the PIN is a unique identifieracross all displays managed by the display control system 940. In stillother examples, the PIN may not be a unique identifier, but instead, adifferent unique identifier may be assigned by either the displaycontrol system 940 and/or the display 930.

The pairing information (e.g., the PIN) may then be displayed on thedisplay as previously described at operation 908. The user at the localcomputing device 920 may then select an option in a communicationapplication to allow a remote user of a content source device (e.g.,remote content source device 910) to share content on the designatedsub-display. The user of the local computing device 920 may enter thesub-display pin into a GUI element of the communication applicationexecuting on the local computing device 920 as well as a designation ofthe content source device or a designation of a remote user. In theexample of designating the remote user (as opposed to the remote contentsource device itself) the system determines a remote content sourcedevice that the user is currently authenticated on.

At operation 912, the communication application on the local computingdevice 920 then sends a message with the PIN (or other sub-displayidentifier), the remote user id (or content source device id), andinformation (e.g., the address) about the display control system 940 tothe communication service 960. The communication service 960 providescommunication functionality (such as chats, video calls, audio calls,online meetings, and the like) to various devices executing thecommunication applications—such as local computing device 920 and remotecontent source device 910.

The communication service 960 receives this message and at operation914, determines an address of the remote content source device 910 thatwas selected by the local computing device 920—e.g., by consulting amapping between remote content source device 910 identifier and/or useridentifiers and network addresses. The network address of the remotecontent source device 910 is then used to send a notification to thecommunication application instance executing on the remote contentsource device 910. The communication application instance executing onthe remote content source device 910 may prompt the user to select anapplication or other content to share (or reject the offer to share ifthe user does not wish to share the content). Once the user of theremote content source device 910 selects the content, at operation 916,the remote content source device 910 may begin streaming the content tothe communication service 960, which may establish a connection with thedisplay control system 940 using the address transmitted by the localcomputing device 920.

At operation 918, the communication service 960 may forward the streamedcontent along with the PIN or other identifier of the sub-displayprovided by the application on the local computing device 920 to thedisplay control system 940. The display control system 940 may thenassociate the communication service 960 with the sub-display identifiedby the PIN or other identifier (e.g., fill out the content source deviceinfo with information on either the remote content source device 910, orthe communication service 960). At operation 918, the content sent fromthe remote content source device 910 may be sent to the display 930where it is displayed in the sub-display.

Thus, content sent from the remote content source device 910, may bedisplayed on the display 930 by going through the communication service960 and the display control system 940. As described, the PIN or someother unique sub-display identifier may be used to associate particularcontent with a particular sub-display. By using the communicationservice 960 as an intermediary, the content may also be sent, inparallel, to other participants of a network-based communication session(e.g., an online meeting) in addition to being displayed on thesub-display of the display 930.

In some examples, rather than have a designation of a remote contentsource device 910, the content currently presented in a network-basedcommunication session (e.g., an online meeting) may automatically besent by the communication service 960 to the display control system 940.The display control system 940 associates the sub-display with thenetwork-based communication session and causes the content sent by thecommunication service 960 to be displayed.

In FIG. 9, the communication service 960 was utilized to facilitate thedisplay of content from the remote content source device 910 to thedisplay 930. In other implementations, the communication service 960 maynot be involved. That is, the message with the PIN and display controlsystem information may be sent directly from the local computing device920 to the remote content source device 910. The content may then bedirectly sent from the remote content source device 910 to the displaycontrol system 940.

By decoupling the content from passing through the local computingdevice 920 once setup is complete, actions taken by the local computingdevice 920 do not affect the content displayed in the sub-display. Localcomputing device 920 could leave the network-based communication sessionand the content would continue to be displayed, so long as thecommunication session continues for other participants.

Input from the sub-display may also be delivered to the remote contentsource device 910. For example, the input may be routed back through thedisplay control system 940, back through the communication service 960,and finally to an application on the remote content source device 910.Thus, input on the sub-display of display 930—entered by local users,may appear on the remote content source device 910. In some examples,the communication service 960 and/or the communication application mayhave various rules on what inputs are allowed and what inputs are notallowed. While the example of FIG. 9 used a communication applicationand a communication service 960, one of ordinary skill in the art withthe benefit of the present disclosure will appreciate that otherapplications and services may be used if they implement thefunctionality of FIG. 9.

As shown, in FIG. 9, the local computing device 920 provided a PIN orother pairing information of the sub-display. In some examples, ratherthan a PIN, the local computing device 920 may provide any pairinginformation. For example, a biometric pairing. That is, the user oflocal computing device 920 may enter their biometric into the pairinginput area of the display. The display may scan the biometric and sendthe biometric scan data to the display control system. The displaycontrol system may store the biometric scan data in the sub-displayinformation 950. The communication service 960 may then pair with thesub-display by sending biometric data of the user of local computingdevice 920. This biometric data may be scanned by the local computingdevice 920 ahead of time or when the sub-display is created. In someexamples, the biometric data of the user of the local computing device920 may be stored at the communication service 960.

As previously described, the local computing device 920 may provide anypairing information. This may be generalized further by stating that thelocal computing device 920 may provide any sub-display identifier.Example sub-display identifiers include PIN numbers, alphanumericsequences, biometric information, sub-display titles, and the like.

Additionally, as described in FIG. 9, the local computing device 920 maysend to the communication service 960 an address of the display controlsystem 940 discovered using a broadcast message (or using some othermethod). In other examples, the communication service 960 may alreadyhave the display control system address. For example, the displaycontrol system 940 may be an in-room meeting room system that mayregister with the communication service 960. In some examples, thecommunication service 960 may be aware of a plurality of display controlsystems 940 and may select the appropriate display control system tosend content to based upon a location of the local computing device 920and the display 930 (and which displays 930 the display control system940 control), a selection by local computing device 920, or the like.

In some examples, the local computing device 920 may not be part of theflow of FIG. 9. That is, a local user may designate the sub-display, andselect the remote content source device from a UI of the display 930.The display 930 may pass the information on the sub-display, includingthe designated remote content source device 910 (or a user of the remotecontent source device 910) to the display control system 940. Displaycontrol system may then send a message (e.g., message 912) to thecommunication service 960. Display control system 940 and communicationservice 960 may be in communication as a result of the display controlsystem 940 being registered with the communication service 960 and/orthe display control system 940 being admitted to a network-basedcommunication (e.g., an online meeting) provided by the communicationservice 960.

Input Sub-Displays

As previously noted, a sub-display may accept input that is forwarded toan application on a content source device that is paired with thesub-display. In some examples, a sub-display may be designated that doesnot display content, but rather is exclusively for accepting input. Insome examples, such a sub-display may show the input (e.g., eitherpermanently or temporarily—such as a for a predetermined time), but inother examples it may not show the input.

FIG. 10 illustrates a display with two designated sub-displays, oneexclusively for input, according to some examples of the presentdisclosure. Sub-display 1040 may be a sub-display that may show contentstreamed by application A 1030 on the content source device 1020. Theapplication A 1030 executes on the content source device 1020. Inputdirected to the sub-display 1040, such as a touch gesture within thesub-display 1040, may be sent to application A 1030. For example, theinput may be an input that is treated by application A 1030 as if it wasan input local to the content source device 1020. An input “local” to adevice is an input produced by an input device coupled through a localinterface (e.g., Universal Serial Bus (USB), Peripheral ComponentInterconnect (PCI), or the like).

In contrast, sub-display 1050 may be an input-only sub-display. That is,input from the sub-display 1050 may be sent to one or more applicationsexecuting on the linked content source device, such as content sourcedevice 1020. This may include application A 1030 or be a differentapplication than application A 1030. In some examples, the inputdirected to the sub-display 1050 may be treated the same way as input tothe sub-display 1040—that is, processed as a local input to applicationA 1030. In other examples, the input to sub-display 1050 may be relatedto the content streamed by application A 1030 that is displayed in thesub-display 1040, but the input may not be an edit to that content. Forexample, the input to the sub-display 1050 may edit content supplementalto the content displayed in sub-display 1040. Supplemental content iscontent that comments on, adds to, or otherwise supplements the contentdisplayed. As a more specific example, a user may display a slide showpresentation in the sub-display 1040 and write notes for each slide inthe sub-display 1050. The notes may be saved in a notes file for eachslide. That is, the notes may be saved in a notes file that correspondsto the specific slide shown in the sub-display 1040. More generally, theinput into the sub-display 1050 may be correlated to the specificcontent shown in the sub-display 1040. Other specific examples includenotes about a video displayed (e.g., the notes being correlated to thevideo based upon a playback time), a document being edited (e.g., theinput being a comment in the document that is inserted at the activepoint of editing), and the like.

In other examples, the content may not relate at all to what isdisplayed in the sub-display 1040. For example, the input directed atthe sub-display 1050 may simply be stored in a file or other storage ofthe content source device 1020. The input directed to the sub-display1050 may be stored in a data storage device 1042 and may be stored withthe content streamed by application A 1030 or separately.

In some examples, the input directed to the sub-display 1050 may bedisplayed in the sub-display 1050. In other examples, the input directedto the sub-display 1050 is not displayed in the sub-display 1050. In theexamples in which the input directed to the sub-display 1050 is notdisplayed in the sub-display 1050, this facilitates candid note takingas the input is not reproduced for public viewing. That is, during apresentation the presenter may record thoughts about the presentationthat are then saved for later use without the audience being able toread the notes.

In some examples, the destination of the input of a sub-display may beuser controllable. For example, once the content source device 1020 islinked to a given sub-display, the user may select from a plurality ofoptions on where to send input on any given sub-display and whether todisplay any input entered. If the input modifies content being shared toa different sub-display (e.g., the input to the sub-display 1050modifies content shared in sub-display 1040), the input may be displayedby virtue of updating the content shared. However, input that does notedit content shared on a different sub-display may either be displayedor may not be displayed—depending on implementation and/or userpreferences. User selections of whether or not to display the input maybe made on the content source device 1020 (e.g., in a user interface ofthe application that communicates with the display control system suchas a streaming application or an operating system) or on the display.For example, selections may be made when the sub-display is setup, in acontext menu of the sub-display, or the like.

To create an input-only sub-display such as input-only sub-display 1050,the user may utilize a different gesture to create the input-onlysub-display, may call up a context menu during creation of thesub-display to indicate that the display is input-only, may change thesub-display to be an input-only sub-display after sub-display creation,and/or the like. The user may also choose whether to display input onthe sub-display. That is, even if the sub-display 1050 does not displaycontent, the system (e.g., the display, the display control system, orthe content source device 1020) may show the input on the sub-display.For example, if the user writes notes in the input only sub-display thesub-display may show the notes. In other examples, the input is notshown on the sub-display.

In order to route the input of either sub-display 1040 or sub-display1050, the content source device 1020 may have a table mapping input fromparticular sub-displays to particular actions. For example, a rule mayspecify that the input directed to sub-display 1040 is to be directed toapplication A 1030 as if it were local input and input directed to thesub-display 1050 is to be directed to a notes file associated with acurrently displayed item of content displayed by application A 1030. Thetable may list sub-display identifiers and where the input is to be sentand/or how the input is to be processed. Upon receipt of input from thedisplay 1010 or the display control system, the identifier of thesub-display the input was received in is used to index the table todetermine how to process the input.

Notifications

In some examples, a sub-display may be configured to display one or morenotifications from the content source device. These notifications may beidentified at the content source device 1020. For example, a new mailnotification for an email account associated with a user of the contentsource device; application notifications; communication notificationssuch as new message notifications; call notifications; and the like.Notifications may be generated by application programs executing on thecontent source device. In some examples, these notifications may not berelated to the display service. The notifications may also be generatedby the application that is sharing content in a sub-display that isdisplaying the notification or may be generated by a differentapplication.

The notifications may be displayed within the sub-display, may bedisplayed in a notification area adjacent to the sub-display, or thelike. FIG. 11 illustrates an example of a sub-display 1110 with anotification area 1130 according to some examples of the presentdisclosure. Notification area 1130 may be a pull out, pop-up, or otherarea. In some examples, the notification area 1130 may not be part ofthe sub-display. In other examples, the notification area 1130 may bepart of the sub-display. Sub-display 1110 shows content for applicationA of content source device 1. Content source device 1 receives orotherwise identifies information that, according to instructions of oneor more applications, generates a notification to the user—e.g., via anotification application programming interface (API) of an operatingsystem of the content source device 1. In some examples, the contentsource device 1 may have rules that specify which notifications are tobe displayed to a user on a remote display and how those notificationsare to be shown. For example, the content source device 1 may provide auser interface which allows users to choose which notifications fromwhich applications are displayed on a remote display. For example, whichnotification types are displayed. These rules may differ from rules usedto determine when a notification is to be generated in the firstinstance.

The contents source device 1 then sends a notification indicator alongwith the content of the notification to the display or display controlsystem along with the sub-display identifier (e.g., the PIN). Thedisplay or display control system then shows the notification—forexample, in the sub-display 1110, in a notification area, such asnotification area 1130. In some examples, the notification area may be avisually distinct area adjacent to and sharing a border with the outlineof the sub-display 1110. The notification area 1130 may not overlap thesub-display 1110. The notification may be displayed until dismissed bythe user through an input (e.g., a gesture of a particular specifiedtype) directed to the notification on the sub-display, dismissed on thecontent source device, or may be displayed for a predetermined period oftime. A content source device may also send a message to the display ordisplay control system to remove or otherwise modify the notification.

In some examples, the notification area 1130 may allow for input, suchas gestures that allow actions to be taken on the notifications. Forexample, a predefined gesture may dismiss the notification—causing it tono longer be displayed. Other gestures may cause the display of theapplication on the content source device that generated the application.Responsive to certain gestures, information about the notification maybe displayed in either the sub-display to which the notification isattached (sub-display 1110 in FIG. 11), a different sub-display, or anew (automatically created) sub-display. For example, if thenotification is a new email notification, then a specific gesture mayopen the email.

In some examples, whether the information on the notification isdisplayed in the sub-display to which the notification is associated, adifferent sub-display, or whether it is displayed in a new sub-displaymay be based upon the gesture. One gesture may open a new sub-displayand another gesture may display the information in the already openedsub-display. For example, if the user touches on the notification anddrags the notification away from the sub-display (e.g., like it is“tearing” the notification away)—the system may open a new sub-displayto display information about the notification. In other examples, if theuser taps the notification, then information on the notification may beopened in the current sub-display (e.g., sub-display 1110).

In some examples, specified gestures may allow for entry of inlinereplies to certain communication-related notifications. For example, agesture may cause a display of an input field to allow a user to entertext to reply to an email or text message in an input box in thenotification area.

Notifications may be content-sensitive in that certain content onnotifications may be blurred, or the notification may not be displayedbased upon the content. For example, if the notification is an email,the user may setup the system such that email from certain users, withcertain keywords, or otherwise matching defined criteria, may beblurred, not shown, or the like.

In addition to receiving a notification indication from the contentsource device, the display and/or display control system may detect anotification window within the content shared by the content sourcedevice. This notification may be split into a notification sub-display.The notification sub-display may be docked to the main sub-display inthe form of a notification area 1130. The display control system maymodify, crop, or otherwise remove the notification from the contentshared so that the notification shown in the notification sub-display isthe only instance of that notification that is shown. The displaycontrol system may detect notifications as features using the methodsdescribed herein for detecting communication modalities.

Different Sub-Displays for Different Modalities

In some examples, the display system may automatically create one ormore sub-displays based upon activities, applications, and/or contextsof applications executing on a particular content source device that ispaired with a sub-display. For example, the content source device may beexecuting a communication application that may provide a plurality ofcommunication modalities. Example modalities include network-basedcommunications such as chat communications, voice communications (e.g.,Voice over Internet Protocol—VoIP), video calls, online meetings, filesharing, group chats, topic-based group chat and the like. Topic-basedgroup chat may be a chat with one or more persons that is organized by aparticular topic or group (e.g., a team).

FIG. 12 shows an example of a graphical user interface (GUI) 1200 of acommunication application according to some examples of the presentdisclosure. Command and search bar 1210 allows users to type in one ormore commands or search for files, communications, application features,and the like. On the left, there are various icons for accessing variouscommunication modalities provided by the communication application.Activity icon 1220, when selected, provides a list of recentcommunications involving the user or groups of which the user is a partof. The chat icon 1230, when selected, provides the user withfunctionality to initiate or participate in text communications with oneor more other users. The teams icon 1240, when selected, provides theuser with functionality for creating, modifying, managing, andparticipating in topical, group-based communications. The meetings icon1245, when selected, provides the user with functionality to create,manage, and participate in network-based communications (e.g.,network-based meetings) where the users can share voice, video, and/orcontent with one or more users and with the display. The calls icon1250, when selected, provides the user with functionality to callanother user using VoIP and/or video calling other users. Files icon1260, when selected, allows users to view the files associated with oneor more other communication modalities available to the user through thecommunication application. For example, such as files exchanged in oneor more teams, in a chat, shared during a call or meeting, or the like.In some examples, the files icon 1260 also allows users to share filesbetween each other. The GUI 1200 shows the meetings icon selected and aGUI of an active meeting with a meeting control bar 1265.

FIG. 13 illustrates a display 1305 with multiple sub-displayscorresponding to multiple communication modalities of the communicationapplication according to some examples of the present disclosure. As canbe appreciated and as disclosed herein, multiple independentsub-displays may display different content from a same application of asame content source device. As shown in FIG. 13, the display 1305 hasmultiple sub-displays that are linked to a same application on a samecontent source device and are displaying different communicationmodalities (e.g., different content) of the application. In FIG. 13, thecommunication application of FIG. 12 is sharing different communicationmodalities to different sub-displays. For example, sub-display 1310 mayshow the contents of a primary display area of a meeting that a user iscurrently engaged in. At the same time, sub-display 1314 may show a chatwindow for a chat session that corresponds to the meeting. The chatsession corresponding to the meeting may be a chat session automaticallysetup that allows meeting participants to have a text chat with eachother simultaneously while content is being presented. Sub-display 1316may display a chat thread that is not associated with the meetingdisplayed in the sub-display 1310. Sub-display 1312 may display one ormore conversations from one or more of the teams of the communicationapplication. In the example of FIG. 13, a conversation from team Y.

The sub-displays showing the different communication modalities may becreated manually—for example, the user may select the differentmodalities within the communication application and link them to thevarious sub-displays using a mechanism shown in FIG. 7, wherein the“app” selection may include the various communication mechanisms. Thus,in addition to a selection of applications (e.g., “APP 1,” “APP 2”, “APP3” as shown in FIG. 7), the selection screen may include variouscommunication modalities such as teams, chats, meetings, and the like.In addition, the user may have an option in a context menu correspondingto the communication modality to share the modality in a sub-display.Upon selecting a particular modality, the communication applicationprompts the user for the pairing information of the sub-display, andthen begins sharing the communication modality on the sub-display.

In other examples, the communication application and/or the displaycontrol system may automatically create one or more sub-displays todisplay one or more of the various communication modalities. Forexample, if a first sub-display is setup that is linked to thecommunication application, other sub-displays may be automaticallycreated and linked to the communication application with eachsub-display displaying a different one of the communication modalitiesprovided by the communication application. In some examples, the systemmay prompt the user before automatically creating one or moresub-displays for the one or more communication modalities, but in otherexamples, the system may automatically create the sub-displays withoutuser confirmation or input.

In some examples, the system may create a sub-display for each offeredcommunication modality provided by the communication application. Inother examples, the system may create sub-displays for only certaincommunication modalities. For example, sub-displays may be created forcommunication modalities based upon usage history. In these examples,sub-displays may be created for a prespecified number x of the mostrecently used modalities. In some examples, x may be based upon theamount of space available on the display. That is, the free space notoccupied by sub-displays may be used to determine how many sub-displaysto open for the communication modalities, along with the size of thesub-displays. The sub-displays may change based upon usage history.Thus, if the top three most recently used communication modalities areshown, the content shown in the three sub-displays may change as usageof the communication modalities changes. Thus, content of a recentlyused communication modality may displace content of a less recently usedcommunication modality in a particular sub-display. In some examples,users may “pin” certain modalities, such that they cannot be displacedby other communication modalities. As usage of the display changes(e.g., users close sub-displays) x may increase as free space on thedisplay increases and additional modalities may be selected for display.

In other examples, other methods of choosing the communicationmodalities may be utilized, such as user preferences. The user may setup one or more rules for sharing one or more of the modalities. Forexample, the user may specify that upon discovering a display controlsystem and linking a first modality, certain other modalities may beautomatically displayed. In some examples, conditions for determinewhich modalities to be displayed may include the manually linked firstmodality (e.g., the first modality displayed may have other associatedmodalities that are to be displayed), communication activity and usagehistory of modalities, location of the user, content of the modality(e.g., the system may exclude content with certain words, images, orsensitive material as defined by the user), and the like.

In some examples, a past sub-display history of the user may be utilizedto automatically select one or more modalities. For example, if the userin the past created three sub-displays and linked modalities A, B, andC; then the system may, upon recognizing that the user has linked asub-display to modality A, automatically create sub-displays and linkmodalities B and C (or prompt the user asking if they would like tocreate sub-displays for modalities B and C).

Predictions of what modalities a user may be interested in sharing on asub-display may be made by one or more machine-learned models that aretrained with training data which may (or may not be) labelled with themodalities that users manually paired with a sub-display. The trainingdata may be per-user, across all users, across groups of users, and thelike. Training data may include communication activity on availablemodalities such as last activity, total activity, frequency of activity,recent activity levels, and the like; content of the modalities such askeywords, images, and the like; type of modalities; and/or the like. Themodel may be a logistic regression, neural network, a decision forest, aboosted decision tree, a support vector machine, a Singular ValueDecomposition (SVD) recommender, or the like. During usage, currentfeature data on the modalities—such as the current communicationactivity, current content of the modalities and the like may be used asinput to the model which may output a list of modalities that may beautomatically created and/or suggested to the user and/or linked toautomatically created sub-displays. The feature data may be currentcommunication activity on available modalities such as last activity,total activity, frequency of activity, recent activity levels, and thelike; content of the modalities such as keywords, images, and the like;type of modalities; and/or the like.

In some examples, sizes of the automatically created sub-displays may bepredetermined or may be determined based upon one or more features. Forexample, each modality may have a particular defined size. In otherexamples, each modality may have a particular minimum and/or maximumsize with the actual size set based upon activity of the modality. Forexample, a chat modality may have a certain minimum size to display thechat. In addition, the size may be set or modified by a communicationhistory of the modality, such as setting the size of a sub-display so asto display all messages received within a defined time period. In someexamples, the size may be based upon an available space on the display.For example, if the display has enough space to open a sub-displaycapable of displaying ten past messages, then the most recently receivedten messages may be displayed.

As noted above the system may suggest adding or automatically addadditional sub-displays to users, for example, to display additionalcommunication modalities. More generally, the same process describedabove may also be applied to other content sources executing on thecontent source device. Content sources may include multiple contentitems on a same application (such as different communicationmodalities), different applications, and the like. For example, thesystem may suggest opening a sub-display for another applicationexecuting on the content source device or may suggest opening asub-display for a different piece of content in a same application.Thus, if a web browser is displaying two videos, a first video beingshared in a sub-display, the system may suggest sharing the second videoin a second sub-display.

FIG. 14 shows an illustration of a suggested additional sub-display 1410according to some examples of the present disclosure. In the example ofFIG. 14, the system may suggest another sub-display 1410 that displays adifferent content source (e.g., a different communication modality) of acommunication application. For example, the user may have alreadydefined sub-displays 1420 which shows meeting content and sub-displays1430 which shows a chat thread. Another chat thread may be suggested bythe system based upon a combination of available sub-display space andadditional content to share. For example, a conversation thread maybecome active by receiving a message in the thread from anotherparticipant or by the user sending a message in the thread.

As shown in FIG. 14, the system may provide the suggestion by showing anoutline or shadow version of the suggested sub-display and may include ashadow version of the content that would be displayed. As in FIG. 14, anarea of the sub-display is designated to obtain approval of the user toaccept the suggested sub-display. In some examples, if the user does notaccept within a threshold period of time it is assumed that the userdoes not accept the suggested sub-display and the suggested sub-display1410 disappears. In other examples, a rejection control is displayedthat allows the user to reject the suggested sub-display. In someexamples, users may turn off these suggestions.

FIG. 15 illustrates a data flow of a suggested sub-display according tosome examples of the present disclosure. Display control system 1540 maydetermine that additional space is available (e.g., a minimum amount ofspace is not currently occupied by a sub-display) on the display 1530for additional sub-displays. The display control system 1540 may send anotification 1502 to the content source device 1520 that additionalspace is available.

In response, the content source device 1520 may determine if anyadditional content is available for display and whether that contentshould be displayed (either automatically or suggested to the user). Thecontent source device 1520 may apply the above-mentioned models, rules,or the like to determine suggested content. The content source device1520 may also apply user selectable settings and rules that may preventcertain content from being displayed or suggested to prevent thesuggestion or display of embarrassing or sensitive content.

If additional content is available that complies with the user'ssettings, the content source device 1520 may begin streaming the contentto the display control system 1540. The display control system 1540 maysend a message 1550 suggesting a sub-display to the display 1530. Themessage may provide coordinates and sizing of the suggested sub-display.The display control system 1540 may create a sub-display record (such asdescribed in FIG. 9) for the suggested sub-display. The sub-displayrecord may have one or more fields that indicate that the sub-display ismerely suggested. The message may also include a preview of the content1547. The display 1530 may display the suggested display as described inFIG. 14.

The display 1530 sends a response 1549 indicating whether the useraccepted the suggested sub-display. If the user accepted the suggestedsub-display, then the sub-display record may be changed to reflect thatit is no longer a suggested sub-display and is an actual sub-display.The display control system 1540 may begin sending the additional content1545 to the display 1530. The display may then cause the sub-display tobe shown with the additional content 1545.

In addition to suggesting content for display in a sub-display, thesystem may suggest increasing or decreasing a size of an already createdsub-display. FIG. 16 shows an illustration of a suggested display sizechange according to some examples of the present disclosure. A user hasstarted to designate a display area 1615 by touching in a firstupper-left corner of a display 1610. As the user slides their finger toa lower right corner 1617 of the sub-display, the system recognizes thatadditional space is available on the display. A pop-up box 1619 may bedisplayed indicating that additional space is available and offering asuggested display size 1620. If the user has not released their finger,they may continue to slide their finger to enlarge the sub-display untilthe suggested size (or some other size) is reached and then, byreleasing their finger, the sub-display may be created as a rectanglewith opposite vertices as the start and end touch points. In otherexamples, the user may release their finger (or use another finger) andtap the pop-up box 1619 to automatically increase the size of thesub-display to match the suggested sub-display size.

In other examples, the sub-display size may be suggested after thesub-display is already created. For example, the sub-display may becreated, and then a prompt may allow a user, by use of an input field(e.g., an onscreen button), to accept a suggested resize of thesub-display. The suggestion to resize a sub-display may be made basedupon available space on the display and/or the suitability of thecurrent size to the displayed content.

For example, if the size of the content is such that scroll bars arepresent or if the content is scaled in the sub-display, and there isenough room on the display to increase the size of the sub-display suchthat the content is no longer scaled or there is no longer a need forscroll bars, then the system may suggest a larger sub-display.

A suggested sub-display increase may be responsive to the displaygaining free space by the closing of one or more sub-displays. Thedisplay and/or display control system may monitor the display andsuggest, based upon new free space and/or the content displayed in aparticular sub-display that one or more sub-displays increase theirsize.

In the previous examples, systems were described that providedsuggestions to users to share additional communication modalities inseparate sub-displays. These suggestions were identified by anapplication executing on the content source device. In additionalexamples, instead of an application on the content source deviceidentifying the additional modalities, the display control system mayidentify separate features (such as separate modalities ornotifications) in shared content provided by a content source device.The display control system may automatically split (with or without userapproval) those modalities into different sub-displays.

For example, a content source device may share a window of acommunication application on a sub-display. The display control systemmay receive this content and determine one or more features of thecontent prior to causing the display of this content. The displaycontrol system may split one or more of those features off into theirown separate and independent sub-displays that are automaticallycreated. The decision may be based upon the detected features, thesettings of the display control system, user settings, and/or the like.For example, the user settings may specify the features that are to besplit into their own sub-displays and/or the conditions for doing so(e.g., using if-then rules). The display control system may also causethe content displayed in the first sub-display to omit a visualrepresentation of features that are split off into their ownsub-displays (e.g., by cropping the feature out). That is, the displaycontrol system may be trained to detect certain features in the contentthat are to be split into an independent sub-display and may not displaythose features in the content in the sub-display that is to display theoriginal content.

In some examples, the features to split may be specified via explicituser preferences. In other examples, past user history may be used todetermine the features that are split. That is, if the user hasfrequently manually split a chat session of a communication applicationoff into its own sub-display, the system may predict that the user willdo the same in this instance and may automatically perform that action.For example, the system may train a machine-learning model based uponthe detected shared content features and labels indicating whether theuser has, in the past, split those features into their own sub-displays.The model may then predict, in a given situation, whether the user islikely to split those features into their own sub-displays. If the useris likely to split those features into their own sub-displays, thesystem may automatically split those features—or ask the user if theywould like the system to split those features. Example machine learningalgorithms may include logistic regression, neural networks, decisionforests, decision trees, support vector machine, and/or the like.

FIG. 43 illustrates a content splitting flow according to some examplesof the present disclosure. A screen of a communications application 4320is shared by a content source device. The screen currently shows anetwork-based meeting with meeting content and a sub-display with a chatthread 4322. The chat thread 4322 may be related to the meeting or maybe a separate chat thread. Display control system 4325 may receive thiscontent. As noted previously the content may be information about thedisplay (e.g., a graphical description of the display, video frames, orthe like), application data used to recreate the display, meetingcontent, chat content, or the like. Display control system 4325 maydetect the chat thread 4322 as a feature of the content and may assignthat chat thread 4322 to be displayed in a second sub-display 4335 onthe display 4310. The rest of the content from the screen of thecommunications application 4320 is shown in sub-display 4330—without thechat thread 4335. For example, the detected chat thread 4322 may becropped from the screen of the communications application 4320 prior todisplay in the sub-display 4330. In other examples, the detected chatthread 4322 may be redacted (e.g., blacked out), removed from thecontent, or otherwise not displayed. In some examples, the detected chatthread 4322 may still be displayed in the sub display 4330.

In examples in which the content is video frame content, the displaycontrol system may utilize computer image recognition algorithms torecognize features (e.g., objects) within one or more video frames orimages. For example, by employing example models of various features todetect one or more features. Features may include communicationmodalities, notifications, different aspects of a graphical userinterface, help tips, comments, redactions, edits, individuals (e.g.,different people displayed in the communication application), and thelike. For example, the system may split off a video feed of a number ofdifferent people or roles (e.g., an active speaker role) of anetwork-based communication such as an online meeting into their ownsub-displays. For example, the system may split video feeds of eachindividual person in a network-based communication into their ownsub-display.

Example algorithms for detecting features in video may include machinelearning approaches utilizing approaches such as Viola-Jones objectdetection framework based on Haar features, Scale-invariant featuretransform (SIFT), or Histogram of oriented gradients (HOG) features todefine features that the system looks for. These techniques may employ asupport vector machine classifier on the video to classify the video andto find the feature. Other example algorithms may include deep learningapproaches such as neural networks or convolutional neural networks.

As previously described the location of the feature may be detected andused as content for an independent sub-display. The size of theindependent sub-display may be set based upon the size of the extractedfeature. The extracted feature may be scaled up and/or down based uponthe available space on the display.

In other examples in which the content is not video-based content, thedisplay control system 4325 may search the content for features that arespecified by user settings, system settings, or otherwise specified forsplitting into a separate sub-display. For example, a description of thecontent that is used to render the content (e.g., content describinguser interface 4320) may be searched for various feature indicators,such as headers or other sections delineating notifications or variouscommunication modalities.

While the above described features that are communication modalities,other features may be utilized, such as notifications. For example, whena user shares their screen and a notification from an application popsup onto their desktop, the display control system may detect thenotification and create a new sub-display. In some examples, the newsub-display may be “docked” to the old sub-display in the form of anotification tab as shown with respect to 1130 of FIG. 11. As per thediscussion therein, the user may provide one or more gestures and inputsto the notification.

In some examples, in the case of communication modalities, thesub-display may be paired with the communication server providing thecommunication service. That is, a device executing an instance of acommunication application associated with the communication service mayinput instructions into the communication application to have thecommunication server providing the communication service (or some otherserver with access to the communication sessions) to pair with asub-display. For example, as in the process of FIG. 9, but without theremote content source and with the communication service 960 taking itsplace. The communication service 960 may then share various contentassociated with a communication session on the sub-display. In theseexamples, the communication service 960 may communicate with the displaycontrol system to open up additional sub-displays based upon variouscommunication modalities that may be of interest to users that can viewthe display. These predictions may be made in the same way with the samemethods and factors as described for the communication applicationinstance making those predictions.

Display Control System

FIG. 17 illustrates a logical diagram of a display control system 1700according to some examples of the present disclosure. Serviceadvertisement component 1710 may provide for broadcasting informationabout the display service provided by the display control system to bothdisplays and content source devices. Display registration component 1705may register displays and create display information data structures1750. Sub-display creation component 1715 may create sub-displayinformation data structures 1770 and assist in creation of asub-display. Content routing component 1720 may receive content androute it to the correct sub-display. Input routing component 1725 mayreceive an input from the sub-display and route it to a particularcontent source device. Sub-display pairing 1730 may provide sub-displaypairing with content source devices. Notification component 1735 mayprovide one or more notifications on the display. Suggested sub-displaycomponent 1740 may suggest one or more additional sub-displays and/orchanging a size or configuration of one or more sub-displays. Each ofthese components will be discussed in greater detail in the followingdiscussion.

FIG. 18 illustrates a method flow of a service advertisement accordingto some examples of the present disclosure. At operation 1805, theadvertisement component 1710 may broadcast a service availabilitymessage to a broadcast address over a network. The service availabilitymessage may contain a network address of the display control system, anindication that the display control system is available for controllinga display, and/or that the display control system is available forproviding sub-display sharing services to content source devices.

Displays that are looking for a display control system may respond withtheir capabilities and a request to be controlled by the display controlsystem 1700. Note that the content source devices need not reply to thebroadcast message as the content source devices only need to communicatewith the display service provided by the display control system whenthey pair with a sub-display. The display registration component 1705may handle responses from the displays. Referring back to FIG. 18, atoperation 1810 the display registration component 1705 may receive aregistration message from a display. The display registration component1705 may register the display at operation 1815 by creating an entry inthe display information data structure 1750 with the displayinformation. At operation 1820 a confirmation message may be sent to thedisplay.

Example display information may include a display identifier 1755 thatis unique across the network, or within the display control system 1700.The display identifier 1755 may be assigned by the display controlsystem 1700, the display, or some other entity. Communications betweenthe display control system 1700 and the display may include the displayidentifier 1755 to allow the display control system 1700 to look up thedisplay information data structure 1750 of the proper display. Otherexample display information may include a display address 1760 that is anetwork address of the display, display capabilities 1765—such asresolution, color capabilities, refresh capabilities, inputcapabilities, input types, and the like. The sub-display informationdata structure 1770 may initially be empty but may hold information onsub-displays once those sub-displays are created.

Displays may send a notification when a sub-display is created. Thesub-display creation component 1715 of the display control system 1700may handle this notification.

FIG. 19 illustrates a method flow for handling the sub-display creationnotification according to some examples of the present disclosure. Atoperation 1905 the display control system 1700 may receive a sub-displaycreation message from a display. The sub-display creation component 1715may handle this message at operation 1910 by determining the displayinformation data structure 1750 associated with the display by utilizingthe display identifier sent by the display in the sub-display creationmessage. At operation 1915, the sub-display creation component 1715 maycreate a sub-display information data structure 1770 within, or linkedwith, the display information structure associated with the display themessage was received from that was determined at operation 1910. Atoperation 1920, the sub-display creation component 1715 may send aconfirmation to the display. FIG. 9 shows an example of sub-displayinformation 950 that may be stored.

Sub-display pairing component 1730 may handle requests to pair fromcontent source devices.

FIG. 21 illustrates a flowchart of a pairing operation according to someexamples of the present disclosure. At operation 2105 the sub-displaypairing component 1730 may receive a pairing request from a contentsource device. At operation 2110, the sub-display pairing component 1730may match the pairing information provided by the display and stored inthe sub-display information data structure 1770 with the pairinginformation sent by the content source device to determine whichsub-display to pair with the content source device. If a match is notfound, an error may be returned to the content source device. If a matchis found, then at operation 2115, the sub-display pairing component 1730may update a sub-display information data structure corresponding to thematched sub-display to store information on the content source device.At operation 2120, the sub-display pairing component 1730 may send aconfirmation to the content source device.

As previously described the pairing information may be a PIN that may becreated by the display and provided by the display to the displaycontrol system in the sub-display creation notification. Alternatively,the display control system 1700 may create the PIN and provide it to thedisplay. PINs created by the display control system 1700 may then bepassed back to the display in a confirmation message of the sub-displaycreation (e.g., operation 1920) to allow the display to display the PINin the pairing tab. The PIN may be a unique identifier used to match thecontent source device with a particular sub-display.

In other examples, the pairing information may be biometric based. Inthese examples, since the PIN number no longer is used to uniquelyidentify the sub-display, the sub-display creation component 1715 mayissue a unique identifier (ID) of the sub-display, store this id in thesub-display information, and provide the id to the display. In examplesin which the sub-display identifier is unique for all sub-displaysmanaged by the display control system 1700, the display control system1700 may then uniquely identify any sub-display by the sub-displayidentifier. In examples in which the sub-display identifier is onlyunique to each display managed by the display control system 1700, thedisplay control system 1700 may uniquely identify any sub-display by thesub-display identifier and the display identifier.

In examples in which the pairing information is a biometric template—asdiscussed in more detail with respect to a pairing tab 730 in FIG. 7—thesub-display creation notification may include first biometric scaninformation and the sub-display identifier. The biometric scaninformation (e.g., a biometric template) may be temporarily stored inthe corresponding sub-display information. The pairing request from thecontent source device may also include second biometric scaninformation, this second biometric scan information may be obtainedduring the pairing procedure at the content source device or may havebeen previously obtained at the content source device—such as during anaccount setup procedure.

In these examples, to match the pairing information received with thesub-display information structure (e.g., operation 2110), the displaycontrol system 1700 may iterate through the display information datastructure 1750 and the sub-display information, checking unpairedsub-displays for stored biometric information that matches with thesecond biometric scan information from the content source device. If amatch is found, the content source device is paired with the sub-displaywith the matching biometric scan information. If a match is not found, afailure message may be sent to the content source device.

Once a matching sub-display is found, the content source device addressmay be stored in the sub-display information and the sub-displayidentification (if it is not PIN based) and/or display identifier may bepassed back to the content source device.

Content routing component 1720 may handle content sent from contentsource devices and cause them to be displayed in a sub-display. Thecontent may include a sub-display identifier and/or a displayidentifier.

FIG. 20 illustrates a flowchart of a method for handling content from acontent source device at a display control system according to someexamples of the present disclosure. At operation 2005 the contentrouting component 1720 may receive content from a content source device.The content may contain a sub-display identifier and/or a displayidentifier that may be used by the display control system to identify adisplay information data structure at operation 2010. If the displayinformation data structure 1750 is found, then the display address maybe determined at operation 2015. If the display information datastructure 1750 is not found, an error message may be returned to thecontent source device. If the display information data structure 1750 isfound and an address is located, then at operation 2020 the content maybe sent to the appropriate sub-display at the located display. In someexamples, the display control system may verify that the source of thecontent is the content source device paired with the sub-display byverifying that the return address of the content matches the contentsource device address in the sub-display information. In some examples,the display control system may verify that the content is directed to avalid sub-display of the display.

Input routing component 1725 may receive input from a display or inputdevice and route it to a content source device.

FIG. 22 illustrates a flowchart of a method of routing input to acontent source device according to some examples of the presentdisclosure. At operation 2205, the input routing component 1725 mayreceive an input indicator regarding a sub-display. The input indicatormay be sent by the display (if the input device is communicativelycoupled to the display) or via an input device (e.g., a camera) that iscoupled to the display control system. The indication may include thetype of input (e.g., gesture type), coordinates of the input within asub-display, the sub-display identifier, the display identifier of thedisplay, and the like to the controller. If the display or the inputdevice is not aware of the coordinates of sub-displays, the indicationmay include coordinates of the input within the display itself that maybe used by the controller to determine the sub-display.

For camera-based gesture recognition to determine a sub-displayidentifier of the input, a sub-display identifier may be displayed by,on, or in the display. A computer vision application may be able to readthe sub-display identifiers of sub-displays on the display from acaptured image of the display. The identifier may be displayedcontinuously so that a human can view and read it, or, may be displayedin a way that a human cannot read it, but a machine vision applicationcan. For example, by displaying the identifier only on a limited numberof consecutive frames—e.g., if the refresh rate of the display is 60frames per second, the identifier may be displayed on only 1 frame persecond. This may be too fast for the human eye to see the identifier,but the identifier would be visible to the camera. In other examples,the camera may determine when a new sub-display is created and capturethe identifier. The camera may then track the location and position ofthe sub-display after creation. Gestures on, or directed at, thesub-display may be captured by the camera, recognized, and thecorresponding sub-display identifier maybe determined and sent to thecontroller.

Similarly, if the display itself detects the input, the display maystore and track the sub-display identifier and the sub-displaycoordinates with respect to the display. When a gesture input is on,within, or directed to, the sub-display, the display may send the inputto the display control system.

Turning back to FIG. 22, at operation 2210 the input routing component1725 may determine, from the information provided by the input indicatorand from the sub-display information data structure 1770, the contentsource device paired with that sub-display and a network address of thecontent source device. At operation 2215, the controller may send theinput to the paired content source device.

Notification component 1735 may provide notification functionality tothe sub-display.

FIG. 23 illustrates a flowchart of a method of providing a notificationto a sub-display according to some examples of the present disclosure.At operation 2305 the notification component 1735 may receive anotification indication from a content source device. The notificationindication may include a sub-display identifier, a display identifier,text of the notification, one or more graphics to display along with thenotification, a graphical layout of the notification and/or the like. Insome examples, at operation 2310, the notification component 1735 maydetermine the display information from the sub-display information. Atoperation 2312, the notification component 1735 may create anotification identifier and store the notification identifier with thesub-display information for the sub-display that the notification is tobe displayed in. In some examples, the notification identifier may besent back to the content source device to be provided for futureinteractions with the notification. This allows for multiplenotifications to be displayed with a single sub-display as well asmodifications of the notifications by content source devices.

The notification component 1735 may, at operation 2315, forward thenotification to the display. This notification may include thenotification identifier. In some examples, the display control system ordisplay may reformat the notification to match the display capabilitiesof the display and the notification area. For example, the notificationmay be resized, the color information may be modified, and the like. Thespecific operations may be applied to the notification based upon aseries of rules. The rules may depend on the notification information,the display information, the available notification area, and the like.For example, IF <notification size> is greater than the <notificationarea size> then <Scale the notification>.

Notification component 1735 may also handle inputs received tonotifications.

FIG. 24 illustrates a flowchart of a notification component 1735handling an input to a notification according to some examples of thepresent disclosure. At operation 2405, the notification component 1735may receive an indication of an input directed to the notification. Theindication of the input directed to the notification may include thetype of input (e.g., gesture type), coordinates of the input within anotification area, the sub-display identifier, the display identifier ofthe display, the notification identifier, and the like to thecontroller. In some examples, the notification component 1735 (or thedisplay) may handle the input directly. For example, the notificationcomponent 1735 may handle an input dismissing the notification byinstructing the display to remove the notification and stop displayingthe notification area. The notification component 1735 may then, atoperation 2410, determine the address of the content source device(e.g., based upon the display and/or sub-display identifier) and atoperation 2415 send the indication to the content source device. Inother examples, the notification component 1735 may send the receivedinput indication received at operation 2405 to the content source devicewithout taking any action. The content source device may then takeaction on the notification (not shown), including dismissing thenotification or other action by sending a notification indication withthe notification identifier specifying the desired action.

Suggested sub-display component 1740 may monitor the available space onone or more displays and may determine suggested increases and/ordecreases for the sub-displays based upon the content shared in thesub-displays. The suggested sub-display component 1740 may identify oneor more content transformations that are used to fit the content intoone or more sub-displays. The suggested sub-display component 1740 maydetermine that there is enough unused space on the display (e.g., basedupon space between neighboring sub-displays) to make one or moresub-displays larger so as to reduce or eliminate transformations appliedto content to fit the content in one or more sub-displays. For example,the suggested sub-display component 1740 may apply one or more rules,such as a decision tree, to determine an optimal sub-display size forcontent for one or more sub-displays. The suggested sub-displaycomponent 1740 may then utilize additional rules to determine a new sizeto suggest to a user. The new size may be sent to the display forsuggestion to the user. If the user accepts, the display and displaycontrol system may update their records of the sub-display with the newsize and update the size on the display. The transformations applied tofit the content in the sub-display may then be adjusted based upon thenew size.

Suggested sub-display component 1740 may also suggest additionalsub-displays. For example, the suggested sub-display component 1740 maysend an indication that space on the sub-display is available to thecontent source device. The content source device may then respond withany additional content that the content source device would like tosuggest to the user, e.g., a preview of the content that is suggested tobe displayed. The suggested sub-display component 1740 forwards this tothe display for display of a visual suggestion to the user. If the userapproves displaying this content, then the indication of acceptance isreceived by the suggested sub-display component 1740 from the display orfrom an input device. This indication may then cause the suggestedsub-display component 1740 to instruct the sub-display creationcomponent 1715 to create a new sub-display and sub-display pairingcomponent 1730 to automatically pair the sub-display with the contentsource device. The pairing may be done automatically and the pairingsteps (e.g., displaying or gathering pairing information) may beskipped. A confirmation may then be sent back to the content sourcedevice—which may begin sharing the content.

FIG. 25 illustrates a logical diagram of a display 2500 according tosome examples of the present disclosure. Display registration component2505 may register the display with a display control system. Sub-displaycreation component 2515 may create a sub-display. Input handler 2525 mayhandle one or more inputs to the display. Notification handler 2540 mayhandle one or more received notifications. Content display 2520 mayhandle displaying content in a particular sub-display. Suggestedsub-display 2530 may handle suggestions for additional sub-displays orchanging size or other parameters of existing sub-displays. Each ofthese components is described in greater detail in the followingdiscussion.

FIG. 26 illustrates a method of registering a display with a controlleraccording to some examples of the present disclosure. The displayregistration component 2505 may monitor for broadcast messages at aparticular network address and/or port at operation 2610. If a servicebroadcast message is received at operation 2620, the display maynegotiate for the display control system to control the display atoperation 2630. This may include sending registration information suchas display capabilities and addresses. The display may store informationon the display control system, such as a network address of the displaycontrol system. While the examples mentioned herein have the displaycontrol system sending the advertisements, in other examples, thedisplay and/or content source devices may discover the display controlsystem by sending their own advertisement and/or discovery messages forwhich the display control system listens for and responds.

Sub-display creation component 2515 may recognize inputs to create oneor more sub-displays and communicate with the display control system tocreate the sub-displays.

FIG. 27 illustrates a flowchart of a method of a sub-display creationperformed by a sub-display creation component 2515 according to someexamples of the present disclosure. At operation 2710 the sub-displaycreation component 2515 may identify a gesture that is indicative of asub-display creation. For example, a particular gesture directed to, oron an area outside of an already existing sub-display. The particulargestures may include a finger down (e.g., a finger placed and held on adisplay) and drag. The path the user's finger takes on the screenspecifies the coordinates of the sub-display. Another gesture mayinclude a first finger down to indicate a first diagonal corner and adrag and finger up (e.g., the user lifts their finger up) wherein thefinger up location is the opposite diagonal corner. The display mayanimate the creation of the sub-display as previously described. Thegesture or other input may be determined by an input device of thedisplay such as a touchscreen, or an input device (such as a camera)communicatively coupled to the display or the controller (which may passthe input to the display).

Upon completion of the gesture, the display may create one or more datastructures to store information about, and to track the sub-display. Insome examples, the data structure may include the coordinates of thesub-display—including the coordinates of the outline, a sub-displayidentifier, and the like. In some examples, once created, thesub-displays may be moved, resized, rotated, closed, and the like. Thedata structures may track where on the display the sub-display is.

At operation 2720, the display may determine pairing information. Forexample, the display may assign a PIN number or other unique identifierto the sub-display. In some examples, the unique identifier of thesub-display is the PIN, but in other examples, an additional sub-displayidentifier may be assigned to the sub-display. In other examples, thedisplay may request and receive a PIN or other identifier from thedisplay control system. At operation 2730, the display may show thepairing information, such as within a pairing tab. In examples in whichthe pairing method is a biometric pairing method, the operation of 2720may simply be determining an identifier for the sub-display andoperation 2730 may be displaying instructions for the user to entertheir biometric. In these examples, another operation may be performedwhereby the display communicates with a biometric scanner to obtain abiometric sample of the user. In some examples, the biometric scannermay be instructed only to scan at certain locations within thedisplay—such as at a pairing tab. The coordinates of the pairing tab maybe provided to the biometric scanner. At operation 2730, the sub-displayinformation along with any pairing information and biometric information(depending on the implementation) may be sent to the display controlsystem. The sub-display information, coordinates, pairing (e.g., the PINand/or sub-display identifier), and biometric information may be storedin a data structure of the display.

Content display component 2520 may determine the correct sub-display forreceived content and cause that content to be displayed within thatsub-display.

FIG. 28 illustrates a flowchart of a method of a content displaycomponent 2520 causing content to be displayed according to someexamples of the present disclosure. At operation 2810, the display mayreceive content e.g., from a content source device and/or a displaycontrol system. The content may include the sub-display identifier(e.g., the pairing PIN or some other identifier that is unique to thedisplay or unique to the display control system). At operation 2820, thesub-display to render the content in may be determined based upon thesub-display identifier received with the content (e.g., in a packetheader, stream identifier, or the like), and address of the contentsource device, or the like. At operation 2830, based upon the records ofthe display for that sub-display, the coordinates of the sub-display maybe determined. In some examples, the content may be reformatted to fitwithin the sub-display. This may include scaling, rotating, translating,stretching, cropping, or otherwise transforming the content. In someexamples, prior to transforming the content and/or displaying thecontent, the content may be decoded. For example, the content may beencoded according to one or more encoding schemes in one or moreencoding formats. The display (or the display control system) may decodethe content into a native format for rendering on the display. Atoperation 2835, the content may be displayed in the sub-display.

Suggested sub-display component 2530 may receive one or more suggestionsfrom the suggested sub-display component 2530 of the controller. Thesuggested sub-display component 2530 may advertise on the display one ormore suggestions for changing a size and/or location of one or moresub-displays and present one or more new sub-display creationsuggestions. Users may accept these suggestions by providing an input todo so. The suggestions may be displayed as transparent, ghost, dotted,or as other visually delineated hints that preview a new size, position,shape, or a new sub-display (including suggested content). The user maytap on a button or other input area to accept the change and may haveanother user interface area for rejecting the suggested change. If theuser accepts the change, the display adjusts its data structure of oneor more sub-displays to reflect the change.

Input handler component 2525 may handle input directed to, in, within,or on, one or more sub-displays.

FIG. 29 illustrates a flowchart of a method of handling input directedto a sub-display according to some examples of the present disclosure.At operation 2910, the display may identify an input directed to, in,within, or on, one or more sub-displays. For example, the display maydetect one or more gestures of a touchscreen component of the display.In other examples, a camera communicatively coupled to the display maysend a notification that a gesture or other input is detected. The typeof input (e.g., type of gesture) and the coordinates of the gesture maybe determined. At operation 2920, the coordinates may be utilized todetermine which sub-display the input pertains to. The display may thentranslate the coordinates of the gesture with respect to the fulldisplay (e.g., a display 120) to relative coordinates within thesub-display. These coordinates, the type of input, the coordinates ofthe input within the sub-display, and the like may be sent to thecontroller at operation 2930.

Notification handler component 2540 may handle notifications sent fromthe content source device and/or the display control system as well ashandling inputs directed to the notifications.

FIG. 30 illustrates a flowchart of a method of a notification handlercomponent 2540 handling a notification message from a content sourcedevice or a display control system according to some examples of thepresent disclosure. At operation 3010 the notification handler component2540 may receive a notification indication from the display controlsystem or a content source device. The notification indication mayinclude content for displaying the notification. The notificationindication may include a notification identifier and a sub-displayidentifier. At operation 3020 the display may identify the sub-display,e.g., from the sub-display identifier. At operation 3030 the display mayrender the notification area (e.g., the notification tab). At operation3035 the notification may be displayed in the notification area.

FIG. 31 illustrates a flowchart of a method of a notification handlercomponent 2540 handling a gesture directed to the notification area,according to some examples of the present disclosure. At operation 3110,the display may identify a gesture or other input directed to, on,within, a notification. At operation 3120 the sub-display that thenotification corresponds to may be identified. Additionally, atoperation 3130, the notification identifier may be determined. These maybe determined by matching the coordinates of the gesture with thecoordinates of active sub-displays and/or active notification areas ofactive sub-displays. At operation 3135, the notification may be sent tothe display control system and/or the content source device.

FIG. 32 illustrates a diagram of a content source device 3210 accordingto some examples of the present disclosure. One or more of thecomponents of FIG. 32 may be part of an application, such as acommunications application. User interface component 3215 may provideone or more user interfaces, such as graphical user interfaces (GUIs) toallow users to select content for sharing, selecting a remote display,and pairing with the remote display. User interface component 3215 mayalso provide GUIs for selecting which notifications to display on theremote display.

Remote display sharing component 3220 may identify content e.g., basedupon the selections of the user through the user interface component3215 and send that content to either the display control system or asub-display of the remote display based upon a pairing operationperformed by the remote display pairing component 3225.

Remote display pairing component 3225 may pair with the remote displayby sending the pairing information (PIN, biometrics, or the like)collected through the user interface component 3215 to either thedisplay control system or the remote display. In some examples, theremote display pairing component 3225 may send one or more biometrictemplates (e.g., fingerprint, eye scans, facial recognition scans, etc.)that were taken by the content source device from the user. Thesetemplates may be taken during an application setup process, prior to thepairing process, or as part of the pairing process. These templates maybe compared by the display control system or the remote display with atemplate taken at the remote display. If the templates match, the remotedisplay control system and/or remote display may pair the content sourcedevice and the sub-display on the remote display.

As previously noted, pairing (i.e., linking) describes a process forassociating the content source device with a particular sub-display toallow the routing of content from the content source device to aparticular sub-display. The process entails matching one or more piecesof pairing information (e.g., a PIN, a biometric template, and the like)entered or shown on the display and one or more pieces of pairinginformation entered on the content source device. The pairing, in someexamples, is a way to provide a unique identifier to the content sourcedevice that can be appended to content to properly route the content tothe correct sub-display.

In other examples, the pairing may additionally create a data entry inthe display control system or the display that identifies thesub-display (e.g., using a PIN or other unique identifier) and thecontent source device (e.g., using an IP Address or other uniqueidentifier) and/or application on the content source device such thatdata from the content source device is displayed in the correctsub-display and so that input from the sub-display can be returned tothe proper content source device and/or application on the contentsource device.

For examples in which a single content source device shares content withmultiple sub-displays (e.g., different applications share differentcontent to different sub-displays), the content source device 3210 maykeep a record of which applications and which content on the contentsource device are associated with each sub-display. The remote displaysharing component 3220 receives content from an application, consults adata structure to determine the appropriate sub-display and the addressof the display or display control system and then sends the content tothat address (along with the identifier of the sub-display).

For input coming back to the content source device, the display inputcomponent 3230 of the content source device 3210 may query the datastructure to determine, based upon the sub-display identifier includedin the input, what application to direct the input to. Display inputcomponent 3230 may then forward that input to the application. In oneexample, the input triggers events and/or subroutines of the applicationthe same way that a same input entered using a local input device woulddo. A local input is any input from an input device connected to thecontent source device through a local bus, e.g., a mouse, touchscreen ofthe content source device, or the like.

Notifications component 3235 may detect one or more notifications fromone or more applications executing on the content source device 3210.The notifications component 3235 may compare the notification type,source application, and/or notification components to a set of one ormore rules that specify notifications that are to be displayed on one ormore sub-displays where content is shared. The notifications may bedisplayed in a notification area of a sub-display. The notifications maynot be from the same application that is displaying content in thesub-display. The notifications may be created by the notificationcomponent, including a graphical layout of the notification andincluding a notification identifier.

Input to one or more of the notifications may be sent by the displayand/or the display control system to the content source device 3210 andmay be handled by the notifications component 3235. The notificationinput indication may include a notification identifier. Notificationscomponent 3235 may utilize the notification identifier to identify thenotification and may process the input. Certain inputs may dismiss thenotification. Certain inputs may launch the application that sent thenotification either at the content source device 3210, in thesub-display whose notification tab displayed the notification (whetheror not the application whose content was displayed in that sub-displaygenerated the notification or not), or launch a new sub-display with thenotification. Certain inputs may generate a quick response input area onthe notification tab. Notifications component 3235 may send aninstruction back to the display and/or display control system to modifyor update the notification. For example, in response to an inputdirected to the notification on the display or on the content sourcedevice.

Display service 3240 may listen for broadcast messages from the displaycontrol system or display advertising the availability of the displayfor sharing in one or more sub-displays. The broadcast messages mayinclude an address of one or more display control systems. The servicemay be local to a particular network. In some examples, multipleservices may be discovered. During pairing, the content source devicemay try pairing with all the discovered display control systems (as thecontent source device may not know which display control system controlsthe desired display where the designated sub-display is located). Itwould be unlikely that the pairing information would match two differentunpaired sub-displays. In the alternative, each display may have aglobally unique identification that may be displayed along with, or aspart of the pairing information that the user may also enter. Thedisplay control system may broadcast the display identifiers of thedisplays managed by the display control system. The remote displaypairing component 3225 may then use this broadcast information to selectthe proper display control system.

FIG. 33 illustrates a flowchart of a method of sharing content on asub-display of a remote display according to some examples of thepresent disclosure. At operation 3310, the content source device mayreceive a selection of content to share from a user. For example,through the user interface provided by the user interface component3215. At operation 3315 the content source device may identifysub-display pairing information. For example, a user may enter a PIN,scan a QR code, enter a password, or provide other information displayedon the display (e.g., in a pairing tab) in a user interface elementprovided by user interface component 3215. In other examples, thecontent source device may identify one or more biometric templates ofthe user—e.g., by retrieving it from a database, taking a new scan ofthe biometric, or the like.

At operation 3320 the remote display pairing component 3225 of thecontent source device may identify the display control system, and theaddress of the display control system. As previously described thedisplay control system information may be obtained by monitoring forbroadcast messages. At operation 3325 the remote display pairingcomponent 3225 may send a pairing request message to the address of thedisplay control system with the pairing information. Once pairing isestablished (e.g., a confirmation message from the display controlsystem) the remote display sharing component 3220 may begin sharingcontent at operation 3330. As previously noted, if multiple displaycontrol systems are discovered, the content source device may sendmultiple pairing requests to multiple display control systems.

FIG. 34 illustrates a flowchart of a method of processing input from asub-display according to some examples of the present disclosure. Atoperation 3410 display input component 3230 may receive an inputnotification from a display control system or display. The inputnotification may include the sub-display identifier of the sub-displaythe input is associated with, the input type, location, and otherinformation. The display input component 3230 may identify theapplication associated with the input at operation 3415. For example, bylooking up a data structure storing an association between anapplication and sub-display identifiers. At operation 3420, the input issent to or otherwise delivered to the application.

FIG. 35 illustrates a flowchart of a method of presenting a notificationto a sub-display according to some examples of the present disclosure.At operation 3510 the notifications component 3235 may detect anotification. For example, an operating system may have a notificationservice that applications send notifications to. Notification component3235 may interface with this component. At operation 3515, thenotification component 3235 may determine whether notification settingsallow sending the notification to the sub-display. If the notificationsettings do not allow sending to one or more sub-displays, thenprocessing of this notification for display on a sub-display mayterminate.

If the notification settings allow sending to one or more sub-displays,then at operation 3520 the notifications component 3235 may select asub-display associated with the content source device. If there are nosub-displays associated with the content source device, then processingof this notification for display on a sub-display may terminate. Ifthere is only a single sub-display associated with the content sourcedevice, then this sub-display may be selected. If there are multiplesub-displays associated with this content source device, then one ormore selection methods may be employed. For example, the content sourcedevice may select a sub-display that the content source device mostrecently received input from. In other examples, a largest sub-displaymay be chosen. In still other examples, the display control system maydecide which sub-display associated with the content source device todisplay the notification with. In yet other examples, user preferencesmay be utilized, such as matching certain notification types withsub-displays showing certain shared content. For example, a sub-displayshowing communication applications may display certaincommunication-related notifications.

Once the sub-display is selected, at operation 3530, the notificationmay be sent to the selected sub-display. In some examples, an identifieris associated with the notification and is sent along with thenotification.

FIG. 36 illustrates a flowchart of a method of handling an input such asa gesture directed to a notification displayed on a sub-display. Atoperation 3610, the notifications component 3235 may receive anindication of an input directed to a notification sent by the contentsource device. The indication, as previously described may includeinformation regarding the input—such as a type of input, a location ofthe input, the notification identifier, and other information. Atoperation 3615 the notification component 3235 may identify theapplication that generated the notification—e.g., by using thenotification identifier. At operation 3620, the input information may besent to the application identified at operation 3615.

FIG. 37 illustrates a flowchart of a method 3700 of designating asub-display according to some examples of the present disclosure. Atoperation 3710, the display control system and/or display may identify agesture input directed to a display surface of a display. The gestureinput may define a geometry of a sub-display on the display. The gestureinput defining a sub-display encompassing different coordinates than apreviously specified sub-display. For example, the display may determinethat an input was directed to a location outside of an existingsub-display and send a notification to the display control system inresponse to detecting a gesture to designate the sub-display. Theindication may occur after the gesture is completed (e.g., thesub-display geometry is defined) or as it occurs.

At operation 3720, responsive to the gesture input, the display controlsystem and/or display may create a sub-display and link the sub-displayto a content source device utilizing a linking procedure as describedherein. For example, one or more components of the system may create adata structure storing information about the sub-display. At operation3730, responsive to creating the sub-display and linking the sub-displaywith the content source device, causing content shared by the contentsource device to be displayed in the sub-display of the display. In someexamples, the content may be shared contemporaneously with differentcontent shared in the previously specified sub-display. For example, thedisplay control system may send the content to the display with thesub-display identifier. The display may receive the content, eitherdirectly from the content source device or from the content sourcedevice via the display control system and may render the content withinthe geometry established at operation 3710.

FIG. 38 illustrates a flowchart of a method 3800 of pairing according tosome examples of the present disclosure. At operation 3810, the displaycontrol system and/or display may identify a gesture input directed tothe display specifying a geometry of a sub-display. For example, thedisplay may send a notification to the display control system inresponse to detecting a gesture to designate the sub-display.

At operation 3820, responsive to the gesture input, the display systemmay create a sub-display based upon the defined geometry. As previouslynoted this may entail creating one or more data structures and in someexamples, visually indicating the sub-display on the display. Atoperation 3830, the display may render and display a pairing informationinput area. For example, a pairing tab 630, 730, or 830 which allows forthe input of pairing information such as biometrics, PIN codes, and thelike.

At operation 3840, the display or other input device may detect input ofpairing information at the pairing information input area. The pairinginformation may include a PIN (e.g., via a reverse pairing processpreviously described wherein the content source device determines a PINand displays it to the user who enters it into the pairing informationinput area), a biometric scan of the user, authentication credentials ofan account of a user, an alphanumeric value, and the like. This pairinginformation may be stored in one or more data structures, such as asub-display information data structure 950 in one or more data storageareas.

At operation 3850, the system may link the sub-display with a contentsource device based upon matching the first pairing information receivedat the pairing input area with second pairing information provided bythe content source device. For example, the content source device maysend a pairing request with the second pairing information. As with thefirst pairing information, the second pairing information may be a PIN,authentication credentials of an account of a user, an alphanumericvalue, biometric scan information from a biometric scan taken by thecontent source device, or the like. For example, the system may searchfor a sub-display information data structure with matching pairinginformation. Once a match is found, the sub-display with the matchinginformation is linked (i.e., paired), with the content source device.

At operation 3860, the display and/or display control system may causecontent sent by the content source device to be displayed in thesub-display. For example, the display control system may send thecontent to the display with the sub-display identifier. The display mayreceive the content, either directly from the content source device orfrom the content source device via the display control system and mayrender the content within the geometry established at operation 3810.

FIG. 39 illustrates a flowchart of a method 3900 of an input sub-displayaccording to some examples of the present disclosure. At operation 3910,the display control system and/or display may identify a gesture inputdirected to the display specifying a geometry of a first sub-display.For example, the display may send a notification to the display controlsystem in response to detecting a gesture to designate the sub-display.At operation 3920, responsive to the gesture input, the display controlsystem and/or display may create the first sub-display. At operation3930, the display and/or display control system may cause content sentby the content source device to be displayed in the first sub-display.For example, the display control system may send the content to thedisplay with the sub-display identifier. The display may receive thecontent, either directly from the content source device or from thecontent source device via the display control system and may render thecontent within the geometry established at operation 3910.

At operation 3940 a second gesture input directed to the displayspecifying a geometry of a second sub-display may be identified aspreviously described for operation 3910. At operation 3950, the secondsub-display may be linked to the content source device. At operation3960, an input may be identified that is directed to the secondsub-display. For example, by comparing coordinates of an input tocoordinates of the sub-displays on the display. At operation 3970, theinput directed to the second sub-display are sent to the content sourcedevice. In some examples, the input may be hidden—that is, it is notdisplayed on the first and/or second sub-display. As previously noted,the input may be to content displayed on the first sub-display or may bedirected to content linked to the content displayed on the firstsub-display. While FIG. 39 showed an example where the creation andpairing of a sub-display that displays content with the remote contentsource device prior to designating and pairing an input sub-display, insome examples, an input-only sub-display may be defined first. In stillother examples, only an input-only sub-display may be defined. That is,the methods disclosed do not require the creation and pairing of asub-display showing content with the remote content source device.

FIG. 40 illustrates a flowchart 4000 of a method of providingnotifications according to some examples of the present disclosure. Atoperation 4010, the display control system and/or display may identify agesture input directed to the display specifying a geometry of a firstsub-display as previously described. At operation 4020, responsive tothe gesture input, the display control system and/or display may createa sub-display and link the sub-display to a content source deviceutilizing a linking (i.e., pairing) procedure as described herein. Atoperation 4030, content sent by the content source device is caused tobe displayed in the sub-display as described elsewhere herein.

At operation 4040, the content source device may send, and the displaycontrol system may receive information about a notification on thecontent source device that, if displayed in the sub-display, wouldotherwise overlap with the content. For example, the content sent by thecontent source device for display in the sub-display maybe a stream ofcontent (e.g., a screenshare) and a notification may pop-up on thescreen. The content source device may detect this (e.g., through an APIof an operating system) and may send a notification to the displaycontrol system. The notification may include content beneath thenotification location (e.g., content obscured by the displayednotification). The content source device may obtain the content beneaththe notification location by accessing the rendering pipeline for thecontent and copying the area beneath where the notification is to bedisplayed—e.g., through an operating system. In some examples, contentsent by the content source device for display in the sub-display mayhave one or more layers, the layers may include a notification layerthat is to be rendered on the content. The notification at operation4040 may indicate to the display control system that there is anotification layer in the content. In still other examples, the contentreceived from the content source device may not be a notification andoperation 4040 is not a message from the content source device, butrather detecting a notification using machine-vision techniques asdescribed herein.

At operation 4045, the display control system may cause display of thenotification received at operation 4040 or within the content in anotification region. The notification region may be another sub-display(which may be automatically created), a notification area, such asnotification area 1130, or the like.

At operation 4050, the sub-display is caused to display the content sentby the content source device without the notification overlapping thecontent. That is, the notification layer may not be shown, thenotification in the content may be replaced by the received content thatis under the notification (e.g., from the computing device), or thelike. In some examples, the display control system or the content sourcedevice may determine the content below a notification based upon contentthat was in that location prior to the notification. In other examples,the notification is not included by the content source device in thecontent sent for display in the sub-display and the notification is notdisplayed therein, but in the notification area (region).

FIG. 41 illustrates a flowchart of a method 4100 of designating asub-display area for a remote content source device that is not inphysical proximity to the display according to some examples of thepresent disclosure. The method of FIG. 41 may be performed by acommunication server providing a communications service (e.g., a serverproviding a MICROSOFT TEAMS® service)(e.g., communication service 960).

At operation 4110, the communication server may receive a first messageover a network. The first message may be received from a computingdevice (e.g., a computing device local to the display) or the displaycontrol system. In some examples, the first message may include asub-display identifier and an identifier of a remote content sourcedevice. In some examples where the first message is not received fromthe display control system and/or where the display control system maynot be already known to the communication server, the first message mayinclude an address of a display control system that may be used by thecommunication server to reach the display control system. In someexamples, the first message links the sub-display with the remotecontent source device. That is, the message may be an indication thatthe display control system has linked the sub-display with the remotecontent source device. In other examples, the message may indirectlycause the linking (through later messaging) of the sub-display with theremote content source device.

As noted, the first message may be received from a computing device(e.g., the computing device of the user that designates the display)that is not the display control system or may be received directly fromthe display control system (e.g., the user that designates thesub-display may also input-through a U/I of the display—the remotecontent source device). In some examples, the identifier of the remotecontent source device may be a user identifier—that is, the remotecontent source device is identified by the user that is authenticated tothat device. This authentication may be determined by the communicationserver that provides the communication service.

At operation 4120, the communication server may send a second message tothe remote content source device. This message may indicate that thesub-display is linked to the remote content source device. The remotecontent source device may receive this message and prompt the user toselect content to share (or decline to share).

At operation 4130, the content is received by the server from the remotecontent source device. At operation 4140, the communication server maycause display of the content in the sub-display by sending the contentover the network to the display control system. The communication servermay include the identifier of the sub-display along with the content. Byincluding the sub-display identifier within the content, thecommunication server is specifying which sub-display to display thiscontent in. Including the sub-display identifier within the content mayinclude placing the sub-display identifier within the content, asmetadata to the content, in a header of the packets sent with thecontent, or the like.

FIG. 41 covers both a scenario such as shown in FIG. 9, as well as ascenario such as where the local computing device 920 is notparticipating in the flow described therein, and instead the message(e.g., message 912) is sent by the display control system 940. Thedisplay control system 940 may be known to the communication service 960as a result of the participation of the display control system 940 inone or more network-based communication sessions.

Content sharing, as used herein refers to the process by which a pieceof content on a first device is shared with another device. This sharingmay be contemporaneous with the presentation of content on the contentsource device such that the presentation of the content on other devicesmay happen within a determined time period after the content isdisplayed on or sent from the content source device. In some examples,content sharing may be content streaming or content casting.

Communication modalities, as used herein, denotes differentcommunication methods. For example, email, text messaging, chatmessaging, video calls, phone calls are all different communicationmodalities. A communication application may offer multiple communicationmodalities, and even may offer multiple communication modalities duringa same communication session.

As used herein, pairing information may be a PIN that is used to referto a sequence of numbers and/or letters that is used to match aparticular sub-display to a particular content source device. In someexamples, the PIN may then be provided with the content sent by theremote content source device to uniquely identify a destinationsub-display. In other examples, once pairing is complete, some otheridentifier may be utilized. For example, the display control system mayhave a mapping between Internet Protocol (IP) addresses of the contentsource devices and sub-displays that was determined through the pairingprocess (e.g., by a request to share content sent by the remote contentsharing device that includes the PIN). In other examples, PINs may ormay not be unique and after pairing a new unique identifier is createdand communicated to content source devices and stored with thesub-display information. The content source devices may attach thisunique identifier to the content.

For example, when utilizing biometric pairing, the display controlsystem, display, and/or content source device may create a uniqueidentifier for the sub-display during pairing that is then used toreference that sub-display when sending content or performing otherdisplay operations. In other examples, as mentioned above, the IPaddress of the content source device is used to link the content sourcedevice with the sub-display and no identifier of the sub-display isneeded.

In examples in which the content source devices send content without asub-display identifier, the display control system may still use aunique sub-display identifier when sending the content to the display toenable the display to display the content in the proper sub-display. Inyet other examples, the display control system may simply specifycoordinates where to display the content on the display.

The present disclosure may also allow for sub-displays for showingcontent and collaborating. For example, a virtual whiteboard where usersmay draw input on the display wall and the input is sent to a contentsource device. Other users of the content source device may draw on thewhiteboard on the content source device and those updates may bereflected on the display. This virtual shared whiteboard may also beutilized when the source device is remote (e.g., FIG. 9).

It will be appreciated by one of ordinary skill in the art with thebenefit of the present disclosure that while various functionalities areascribed to particular devices and to particular components or moduleswithin those devices that such descriptions are exemplary. Otherconfigurations are possible and contemplated such that the functionalityascribed to a particular device and/or component may be performed by adifferent device and/or component.

FIG. 42 illustrates a block diagram of an example machine 4200 uponwhich any one or more of the techniques (e.g., methodologies) discussedherein may be implemented. In alternative embodiments, the machine 4200may operate as a standalone device or may be connected (e.g., networked)to other machines. In a networked deployment, the machine 4200 mayoperate in the capacity of a server machine, a client machine, or bothin server-client network environments. In an example, the machine 4200may act as a peer machine in peer-to-peer (P2P) (or other distributed)network environment. The machine 4200 may be a display, display controlsystem, content source device, remote content source device, computingdevice, personal computer (PC), a tablet PC, a set-top box (STB), apersonal digital assistant (PDA), a mobile telephone, a smart phone, aweb appliance, a network router, switch or bridge, or any machinecapable of executing instructions (sequential or otherwise) that specifyactions to be taken by that machine. Machine 4200 may be configured toimplement the processes, create the GUIs, and be configured with thecomponents of FIGS. 1-41 and 43. Further, while only a single machine isillustrated, the term “machine” shall also be taken to include anycollection of machines that individually or jointly execute a set (ormultiple sets) of instructions to perform any one or more of themethodologies discussed herein, such as cloud computing, software as aservice (SaaS), other computer cluster configurations.

Examples, as described herein, may include, or may operate on, logic ora number of components, modules, or mechanisms (hereinafter “modules”).Modules are tangible entities (e.g., hardware) capable of performingspecified operations and may be configured or arranged in a certainmanner. In an example, circuits may be arranged (e.g., internally orwith respect to external entities such as other circuits) in a specifiedmanner as a module. In an example, the whole or part of one or morecomputer systems (e.g., a standalone, client or server computer system)or one or more hardware processors may be configured by firmware orsoftware (e.g., instructions, an application portion, or an application)as a module that operates to perform specified operations. In anexample, the software may reside on a machine readable medium. In anexample, the software, when executed by the underlying hardware of themodule, causes the hardware to perform the specified operations.

Accordingly, the term “module” is understood to encompass a tangibleentity, be that an entity that is physically constructed, specificallyconfigured (e.g., hardwired), or temporarily (e.g., transitorily)configured (e.g., programmed) to operate in a specified manner or toperform part or all of any operation described herein. Consideringexamples in which modules are temporarily configured, each of themodules need not be instantiated at any one moment in time. For example,where the modules comprise a general-purpose hardware processorconfigured using software, the general-purpose hardware processor may beconfigured as respective different modules at different times. Softwaremay accordingly configure a hardware processor, for example, toconstitute a particular module at one instance of time and to constitutea different module at a different instance of time.

Machine (e.g., computer system) 4200 may include a hardware processor4202 (e.g., a central processing unit (CPU), a graphics processing unit(GPU), a hardware processor core, or any combination thereof), a mainmemory 4204 and a static memory 4206, some or all of which maycommunicate with each other via an interlink (e.g., bus) 4208. Themachine 4200 may further include a display unit 4210, an alphanumericinput device 4212 (e.g., a keyboard), and a user interface (UI)navigation device 4214 (e.g., a mouse). In an example, the display unit4210, input device 4212 and UI navigation device 4214 may be a touchscreen display. The machine 4200 may additionally include a storagedevice (e.g., drive unit) 4216, a signal generation device 4218 (e.g., aspeaker), a network interface device 4220, and one or more sensors 4221,such as a global positioning system (GPS) sensor, compass,accelerometer, or other sensor. The machine 4200 may include an outputcontroller 4228, such as a serial (e.g., universal serial bus (USB),parallel, or other wired or wireless (e.g., infrared(IR), near fieldcommunication (NFC), etc.) connection to communicate or control one ormore peripheral devices (e.g., a printer, card reader, etc.).

The storage device 4216 may include a machine readable medium 4222 onwhich is stored one or more sets of data structures or instructions 4224(e.g., software) embodying or utilized by any one or more of thetechniques or functions described herein. The instructions 4224 may alsoreside, completely or at least partially, within the main memory 4204,within static memory 4206, or within the hardware processor 4202 duringexecution thereof by the machine 4200. In an example, one or anycombination of the hardware processor 4202, the main memory 4204, thestatic memory 4206, or the storage device 4216 may constitute machinereadable media.

While the machine readable medium 4222 is illustrated as a singlemedium, the term “machine readable medium” may include a single mediumor multiple media (e.g., a centralized or distributed database, and/orassociated caches and servers) configured to store the one or moreinstructions 4224.

The term “machine readable medium” may include any medium that iscapable of storing, encoding, or carrying instructions for execution bythe machine 4200 and that cause the machine 4200 to perform any one ormore of the techniques of the present disclosure, or that is capable ofstoring, encoding or carrying data structures used by or associated withsuch instructions. Non-limiting machine readable medium examples mayinclude solid-state memories, and optical and magnetic media. Specificexamples of machine readable media may include: non-volatile memory,such as semiconductor memory devices (e.g., Electrically ProgrammableRead-Only Memory (EPROM), Electrically Erasable Programmable Read-OnlyMemory (EEPROM)) and flash memory devices; magnetic disks, such asinternal hard disks and removable disks; magneto-optical disks; RandomAccess Memory (RAM); Solid State Drives (SSD); and CD-ROM and DVD-ROMdisks. In some examples, machine readable media may includenon-transitory machine readable media. In some examples, machinereadable media may include machine readable media that is not atransitory propagating signal.

The instructions 4224 may further be transmitted or received over acommunications network 4226 using a transmission medium via the networkinterface device 4220. The Machine 4200 may communicate with one or moreother machines utilizing any one of a number of transfer protocols(e.g., frame relay, internet protocol (IP), transmission controlprotocol (TCP), user datagram protocol (UDP), hypertext transferprotocol (HTTP), etc.). Example communication networks may include alocal area network (LAN), a wide area network (WAN), a packet datanetwork (e.g., the Internet), mobile telephone networks (e.g., cellularnetworks), Plain Old Telephone (POTS) networks, and wireless datanetworks (e.g., Institute of Electrical and Electronics Engineers (IEEE)802.11 family of standards known as Wi-Fi®, IEEE 802.16 family ofstandards known as WiMax®), IEEE 802.15.4 family of standards, a LongTerm Evolution (LTE) family of standards, a Universal MobileTelecommunications System (UMTS) family of standards, peer-to-peer (P2P)networks, among others. In an example, the network interface device 4220may include one or more physical jacks (e.g., Ethernet, coaxial, orphone jacks) or one or more antennas to connect to the communicationsnetwork 4226. In an example, the network interface device 4220 mayinclude a plurality of antennas to wirelessly communicate using at leastone of single-input multiple-output (SIMO), multiple-inputmultiple-output (MIMO), or multiple-input single-output (MISO)techniques. In some examples, the network interface device 4220 maywirelessly communicate using Multiple User MIMO techniques.

OTHER NOTES AND EXAMPLES

Example 1 is a method of designating a sub-display area, the methodcomprising: using one or more hardware processors: identifying a gestureinput directed to a display surface of a display, the gesture inputdefining a geometry of a sub-display on the display, the sub-displayencompassing different coordinates than a previously specifiedsub-display; responsive to identifying the gesture input, creating thesub-display based upon the geometry and linking the sub-display with acontent source device; and responsive to creating the sub-display andlinking the sub-display with the content source device, causing contentshared by the content source device to be displayed in the sub-displayof the display contemporaneously with different content shared in thepreviously specified sub-display.

In Example 2, the subject matter of Example 1 includes, wherein thecontent shared in the previously specified sub-display is shared by asecond content source device.

In Example 3, the subject matter of Examples 1-2 includes, wherein themethod further comprises: identifying a corrected geometric shape basedupon a first shape produced by the geometry of the sub-display, thecorrected geometric shape most similar of a plurality of correctedgeometric shapes to the first shape; and wherein creating thesub-display based upon the geometry comprises creating the sub-displayusing the corrected geometric shape.

In Example 4, the subject matter of Example 3 includes, wherein thecorrected geometric shape is a polygon with more than four sides.

In Example 5, the subject matter of Examples 3-4 includes, wherein thecorrected geometric shape is a circle.

In Example 6, the subject matter of Examples 3-5 includes, whereinidentifying the corrected geometric shape occurs as the gesture input isin progress, and wherein the method comprises updating an outline of thegesture input to match the corrected geometric shape.

In Example 7, the subject matter of Examples 1-6 includes, wherein thedifferent content shared in the previously specified sub-display isshared by the content source device.

In Example 8, the subject matter of Examples 1-7 includes, identifyingan input directed to the display surface of the display in a locationcorresponding to the sub-display; and responsive to determining that theinput is directed to the sub-display, sending the input to the contentsource device based upon the linking of the sub-display and the contentsource device.

In Example 9, the subject matter of Example 8 includes, wherein sendingthe input to the content source device comprises sending an identifierof the sub-display to the content source device, the content sourcedevice routing the input to an application based upon the sub-displayidentifier.

In Example 10, the subject matter of Examples 1-9 includes, identifyinga first and second simultaneous input, both directed to the displaysurface; determining that the first simultaneous input is directed tothe sub-display; responsive to determining that the first simultaneousinput is directed to the sub-display, sending the first simultaneousinput to the content source device; determining that the secondsimultaneous input is directed to the previously specified sub-display;and responsive to determining that the second simultaneous input isdirected to the previously specified sub-display, sending the secondsimultaneous input to a second content source device linked to thepreviously specified sub-display.

In Example 11, the subject matter of Example 10 includes, whereindetermining that the first simultaneous input is directed to thesub-display comprises determining that a coordinate of the firstsimultaneous input on the display corresponds to a coordinate belongingto the sub-display.

In Example 12, the subject matter of Examples 1-11 includes, causingdisplay of a geometric shape that defines boundaries of the sub-display.

In Example 13, the subject matter of Examples 1-12 includes, whereinresponsive to creating the sub-display and linking the sub-display withthe content source device, causing content shared by the content sourcedevice to be displayed in the sub-display of the display comprises:providing an address of the display to the content source device, andwherein the content source device sends the content to the address.

In Example 14, the subject matter of Examples 1-13 includes, whereinresponsive to creating the sub-display and linking the sub-display withthe content source device, causing content shared by the content sourcedevice to be displayed in the sub-display of the display comprises:receiving the content from the content source device; and sending thecontent to the display.

In Example 15, the subject matter of Examples 1-14 includes, whereinresponsive to creating the sub-display and linking the sub-display withthe content source device, causing content shared by the content sourcedevice to be displayed in the sub-display of the display comprises:receiving the content from the content source device; and rendering thecontent on the display in the sub-display.

In Example 16, the subject matter of Examples 1-15 includes, wherein theone or more hardware processors are communicatively coupled through alocal bus with the display.

In Example 17, the subject matter of Examples 1-16 includes, wherein theone or more hardware processors are communicatively coupled to thedisplay through a packet-based Internet Protocol network.

In Example 18, the subject matter of Examples 1-17 includes, wherein thedisplay is a touchscreen display and wherein the gesture input is atouch input on the display surface.

In Example 19, the subject matter of Examples 1-18 includes, wherein thedisplay surface is a projection screen and the display is provided by aprojector.

In Example 20, the subject matter of Examples 1-19 includes, displayinga prompt on the display, the prompt indicating that the display hasadditional free space not occupied by sub-displays, the prompt offeringa user a selection to automatically increase a size of the sub-displayto fill the additional free space; and responsive to receiving anindication to increase the size of the sub-display, increasing the sizeof the sub-display.

In Example 21, the subject matter of Example 20 includes, wherein theprompt is displayed based upon a determination that the content is beingscaled to fit in the sub-display.

In Example 22, the subject matter of Examples 1-21 includes, wherein thecontent is content of a first source and the different content from asecond source, and wherein the method further comprises: displaying aprompt on the display, the prompt suggesting automatically creating anew sub-display with content from a third source, the first and thirdsources both from the content source device; and responsive to receivingan indication to create the new sub-display, creating the newsub-display and displaying the content from the third source.

In Example 23, the subject matter of Example 22 includes, wherein thefirst and third sources are different communication types provided by acommunication application executing on the content source device.

In Example 24, the subject matter of Examples 22-23 includes, whereindisplaying the prompt is responsive to determining that the display hasadditional free space not occupied by sub-displays and that the contentsource device includes the third content source not displayed in asub-display.

In Example 25, the subject matter of Examples 1-24 includes, whereinidentifying the gesture input comprises receiving an indication of thegesture input from the display.

In Example 26, the subject matter of Examples 1-25 includes, whereinidentifying the gesture input comprises detecting the gesture input at atouchscreen of the display.

Example 27 is a computing device for designating a sub-display area, thedevice comprising: one or more hardware processors; a memory, comprisinginstructions, which when executed by the one or more hardwareprocessors, cause the one or more hardware processors to performoperations comprising: identifying a gesture input directed to a displaysurface of a display, the gesture input defining a geometry of asub-display on the display, the sub-display encompassing differentcoordinates than a previously specified sub-display; responsive toidentifying the gesture input, creating the sub-display based upon thegeometry and linking the sub-display with a content source device; andresponsive to creating the sub-display and linking the sub-display withthe content source device, causing content shared by the content sourcedevice to be displayed in the sub-display of the displaycontemporaneously with different content shared in the previouslyspecified sub-display.

In Example 28, the subject matter of Example 27 includes, wherein thecontent shared in the previously specified sub-display is shared by asecond content source device.

In Example 29, the subject matter of Examples 27-28 includes, whereinthe operations further comprise: identifying a corrected geometric shapebased upon a first shape produced by the geometry of the sub-display,the corrected geometric shape most similar of a plurality of correctedgeometric shapes to the first shape; and wherein creating thesub-display based upon the geometry comprises creating the sub-displayusing the corrected geometric shape.

In Example 30, the subject matter of Example 29 includes, wherein thecorrected geometric shape is a polygon with more than four sides.

In Example 31, the subject matter of Examples 29-30 includes, whereinthe corrected geometric shape is a circle.

In Example 32, the subject matter of Examples 29-31 includes, whereinthe operations of identifying the corrected geometric shape occurs asthe gesture input is in progress, and wherein the operations furthercomprise updating an outline of the gesture input to match the correctedgeometric shape.

In Example 33, the subject matter of Examples 27-32 includes, whereinthe different content shared in the previously specified sub-display isshared by the content source device.

In Example 34, the subject matter of Examples 27-33 includes, whereinthe operations further comprise: identifying an input directed to thedisplay surface of the display in a location corresponding to thesub-display; and responsive to determining that the input is directed tothe sub-display, sending the input to the content source device basedupon the linking of the sub-display and the content source device.

In Example 35, the subject matter of Example 34 includes, wherein theoperations of sending the input to the content source device comprisessending an identifier of the sub-display to the content source device,the content source device routing the input to an application based uponthe sub-display identifier.

In Example 36, the subject matter of Examples 27-35 includes, whereinthe operations further comprise: identifying a first and secondsimultaneous input, both directed to the display surface; determiningthat the first simultaneous input is directed to the sub-display;responsive to determining that the first simultaneous input is directedto the sub-display, sending the first simultaneous input to the contentsource device; determining that the second simultaneous input isdirected to the previously specified sub-display; and responsive todetermining that the second simultaneous input is directed to thepreviously specified sub-display, sending the second simultaneous inputto a second content source device linked to the previously specifiedsub-display.

In Example 37, the subject matter of Example 36 includes, wherein theoperations of determining that the first simultaneous input is directedto the sub-display comprises determining that a coordinate of the firstsimultaneous input on the display corresponds to a coordinate belongingto the sub-display.

In Example 38, the subject matter of Examples 27-37 includes, whereinthe operations further comprise: causing display of a geometric shapethat defines boundaries of the sub-display.

In Example 39, the subject matter of Examples 27-38 includes, whereinthe operations of responsive to creating the sub-display and linking thesub-display with the content source device, causing content shared bythe content source device to be displayed in the sub-display of thedisplay comprises: providing an address of the display to the contentsource device, and wherein the content source device sends the contentto the address.

In Example 40, the subject matter of Examples 27-39 includes, whereinthe operations of responsive to creating the sub-display and linking thesub-display with the content source device, causing content shared bythe content source device to be displayed in the sub-display of thedisplay comprises: receiving the content from the content source device;and sending the content to the display.

In Example 41, the subject matter of Examples 27-40 includes, whereinthe operations of responsive to creating the sub-display and linking thesub-display with the content source device, causing content shared bythe content source device to be displayed in the sub-display of thedisplay comprises: receiving the content from the content source device;and rendering the content on the display in the sub-display.

In Example 42, the subject matter of Examples 27-41 includes, whereinthe one or more hardware processors are communicatively coupled througha local bus with the display.

In Example 43, the subject matter of Examples 27-42 includes, whereinthe one or more hardware processors are communicatively coupled to thedisplay through a packet-based Internet Protocol network.

In Example 44, the subject matter of Examples 27-43 includes, whereinthe display is a touchscreen display and wherein the gesture input is atouch input on the display surface.

In Example 45, the subject matter of Examples 27-44 includes, whereinthe display surface is a projection screen and the display is providedby a projector.

In Example 46, the subject matter of Examples 27-45 includes, whereinthe operations further comprise: displaying a prompt on the display, theprompt indicating that the display has additional free space notoccupied by sub-displays, the prompt offering a user a selection toautomatically increase a size of the sub-display to fill the additionalfree space; and responsive to receiving an indication to increase thesize of the sub-display, increasing the size of the sub-display.

In Example 47, the subject matter of Example 46 includes, wherein theprompt is displayed based upon a determination that the content is beingscaled to fit in the sub-display.

In Example 48, the subject matter of Examples 27-47 includes, whereinthe content is content of a first source and the different content froma second source, and wherein the operations further comprise: displayinga prompt on the display, the prompt suggesting automatically creating anew sub-display with content from a third source, the first and thirdsources both from the content source device; and responsive to receivingan indication to create the new sub-display, creating the newsub-display and displaying the content from the third source.

In Example 49, the subject matter of Example 48 includes, wherein thefirst and third sources are different communication types provided by acommunication application executing on the content source device.

In Example 50, the subject matter of Examples 48-49 includes, whereinthe operations of displaying the prompt is responsive to determiningthat the display has additional free space not occupied by sub-displaysand that the content source device includes the third content source notdisplayed in a sub-display.

In Example 51, the subject matter of Examples 27-50 includes, whereinthe operations of identifying the gesture input comprises receiving anindication of the gesture input from the display.

In Example 52, the subject matter of Examples 27-51 includes, whereinthe operations of identifying the gesture input comprises detecting thegesture input at a touchscreen of the display.

Example 53 is a machine-readable medium, storing instructions fordesignating a sub-display area, the instructions, when executed by amachine, cause the machine to perform operations comprising: identifyinga gesture input directed to a display surface of a display, the gestureinput defining a geometry of a sub-display on the display, thesub-display encompassing different coordinates than a previouslyspecified sub-display; responsive to identifying the gesture input,creating the sub-display based upon the geometry and linking thesub-display with a content source device; and responsive to creating thesub-display and linking the sub-display with the content source device,causing content shared by the content source device to be displayed inthe sub-display of the display contemporaneously with different contentshared in the previously specified sub-display.

In Example 54, the subject matter of Example 53 includes, wherein thecontent shared in the previously specified sub-display is shared by asecond content source device.

In Example 55, the subject matter of Examples 53-54 includes, whereinthe operations further comprise: identifying a corrected geometric shapebased upon a first shape produced by the geometry of the sub-display,the corrected geometric shape most similar of a plurality of correctedgeometric shapes to the first shape; and wherein creating thesub-display based upon the geometry comprises creating the sub-displayusing the corrected geometric shape.

In Example 56, the subject matter of Example 55 includes, wherein thecorrected geometric shape is a polygon with more than four sides.

In Example 57, the subject matter of Examples 55-56 includes, whereinthe corrected geometric shape is a circle.

In Example 58, the subject matter of Examples 55-57 includes, whereinthe operations of identifying the corrected geometric shape occurs asthe gesture input is in progress, and wherein the operations furthercomprise updating an outline of the gesture input to match the correctedgeometric shape.

In Example 59, the subject matter of Examples 53-58 includes, whereinthe different content shared in the previously specified sub-display isshared by the content source device.

In Example 60, the subject matter of Examples 53-59 includes, whereinthe operations further comprise: identifying an input directed to thedisplay surface of the display in a location corresponding to thesub-display; and responsive to determining that the input is directed tothe sub-display, sending the input to the content source device basedupon the linking of the sub-display and the content source device.

In Example 61, the subject matter of Example 60 includes, wherein theoperations of sending the input to the content source device comprisessending an identifier of the sub-display to the content source device,the content source device routing the input to an application based uponthe sub-display identifier.

In Example 62, the subject matter of Examples 53-61 includes, whereinthe operations further comprise: identifying a first and secondsimultaneous input, both directed to the display surface; determiningthat the first simultaneous input is directed to the sub-display;responsive to determining that the first simultaneous input is directedto the sub-display, sending the first simultaneous input to the contentsource device; determining that the second simultaneous input isdirected to the previously specified sub-display; and responsive todetermining that the second simultaneous input is directed to thepreviously specified sub-display, sending the second simultaneous inputto a second content source device linked to the previously specifiedsub-display.

In Example 63, the subject matter of Example 62 includes, wherein theoperations of determining that the first simultaneous input is directedto the sub-display comprises determining that a coordinate of the firstsimultaneous input on the display corresponds to a coordinate belongingto the sub-display.

In Example 64, the subject matter of Examples 53-63 includes, whereinthe operations further comprise: causing display of a geometric shapethat defines boundaries of the sub-display.

In Example 65, the subject matter of Examples 53-64 includes, whereinthe operations of responsive to creating the sub-display and linking thesub-display with the content source device, causing content shared bythe content source device to be displayed in the sub-display of thedisplay comprises: providing an address of the display to the contentsource device, and wherein the content source device sends the contentto the address.

In Example 66, the subject matter of Examples 53-65 includes, whereinthe operations of responsive to creating the sub-display and linking thesub-display with the content source device, causing content shared bythe content source device to be displayed in the sub-display of thedisplay comprises: receiving the content from the content source device;and sending the content to the display.

In Example 67, the subject matter of Examples 53-66 includes, whereinthe operations of responsive to creating the sub-display and linking thesub-display with the content source device, causing content shared bythe content source device to be displayed in the sub-display of thedisplay comprises: receiving the content from the content source device;and rendering the content on the display in the sub-display.

In Example 68, the subject matter of Examples 53-67 includes, whereinthe machine is communicatively coupled through a local bus with thedisplay.

In Example 69, the subject matter of Examples 53-68 includes, whereinthe machine is communicatively coupled to the display through apacket-based Internet Protocol network.

In Example 70, the subject matter of Examples 53-69 includes, whereinthe display is a touchscreen display and wherein the gesture input is atouch input on the display surface.

In Example 71, the subject matter of Examples 53-70 includes, whereinthe display surface is a projection screen and the display is providedby a projector.

In Example 72, the subject matter of Examples 53-71 includes, whereinthe operations further comprise: displaying a prompt on the display, theprompt indicating that the display has additional free space notoccupied by sub-displays, the prompt offering a user a selection toautomatically increase a size of the sub-display to fill the additionalfree space; and responsive to receiving an indication to increase thesize of the sub-display, increasing the size of the sub-display.

In Example 73, the subject matter of Example 72 includes, wherein theprompt is displayed based upon a determination that the content is beingscaled to fit in the sub-display.

In Example 74, the subject matter of Examples 53-73 includes, whereinthe content is content of a first source and the different content froma second source, and wherein the operations further comprise: displayinga prompt on the display, the prompt suggesting automatically creating anew sub-display with content from a third source, the first and thirdsources both from the content source device; and responsive to receivingan indication to create the new sub-display, creating the newsub-display and displaying the content from the third source.

In Example 75, the subject matter of Example 74 includes, wherein thefirst and third sources are different communication types provided by acommunication application executing on the content source device.

In Example 76, the subject matter of Examples 74-75 includes, whereinthe operations of displaying the prompt is responsive to determiningthat the display has additional free space not occupied by sub-displaysand that the content source device includes the third content source notdisplayed in a sub-display.

In Example 77, the subject matter of Examples 53-76 includes, whereinthe operations of identifying the gesture input comprises receiving anindication of the gesture input from the display.

In Example 78, the subject matter of Examples 53-77 includes, whereinthe operations of identifying the gesture input comprises detecting thegesture input at a touchscreen of the display.

Example 79 is a device for designating a sub-display area, the devicecomprising: means for identifying a gesture input directed to a displaysurface of a display, the gesture input defining a geometry of asub-display on the display, the sub-display encompassing differentcoordinates than a previously specified sub-display; means for creating,responsive to identifying the gesture input, the sub-display based uponthe geometry and linking the sub-display with a content source device;and means for causing content shared by the content source device to bedisplayed in the sub-display of the display contemporaneously withdifferent content shared in the previously specified sub-displayresponsive to creating the sub-display and linking the sub-display withthe content source device.

In Example 80, the subject matter of Example 79 includes, wherein thecontent shared in the previously specified sub-display is shared by asecond content source device.

In Example 81, the subject matter of Examples 79-80 includes, whereinthe device further comprises: means for identifying a correctedgeometric shape based upon a first shape produced by the geometry of thesub-display, the corrected geometric shape most similar of a pluralityof corrected geometric shapes to the first shape; and wherein creatingthe sub-display based upon the geometry comprises creating thesub-display using the corrected geometric shape.

In Example 82, the subject matter of Example 81 includes, wherein thecorrected geometric shape is a polygon with more than four sides.

In Example 83, the subject matter of Examples 81-82 includes, whereinthe corrected geometric shape is a circle.

In Example 84, the subject matter of Examples 81-83 includes, whereinthe means for identifying the corrected geometric shape occurs as thegesture input is in progress, and wherein the device further comprisesmeans for updating an outline of the gesture input to match thecorrected geometric shape.

In Example 85, the subject matter of Examples 79-84 includes, whereinthe different content shared in the previously specified sub-display isshared by the content source device.

In Example 86, the subject matter of Examples 79-85 includes, means foridentifying an input directed to the display surface of the display in alocation corresponding to the sub-display; and means for sending theinput to the content source device based upon the linking of thesub-display and the content source device responsive to determining thatthe input is directed to the sub-display.

In Example 87, the subject matter of Example 86 includes, wherein themeans for sending the input to the content source device comprises meansfor sending an identifier of the sub-display to the content sourcedevice, the content source device routing the input to an applicationbased upon the sub-display identifier.

In Example 88, the subject matter of Examples 79-87 includes, means foridentifying a first and second simultaneous input, both directed to thedisplay surface; means for determining that the first simultaneous inputis directed to the sub-display; means for sending the first simultaneousinput to the content source device responsive to determining that thefirst simultaneous input is directed to the sub-display; means fordetermining that the second simultaneous input is directed to thepreviously specified sub-display; and means for sending the secondsimultaneous input to a second content source device linked to thepreviously specified sub-display responsive to determining that thesecond simultaneous input is directed to the previously specifiedsub-display.

In Example 89, the subject matter of Example 88 includes, wherein themeans for determining that the first simultaneous input is directed tothe sub-display comprises means for determining that a coordinate of thefirst simultaneous input on the display corresponds to a coordinatebelonging to the sub-display.

In Example 90, the subject matter of Examples 79-89 includes, means forcausing display of a geometric shape that defines boundaries of thesub-display.

In Example 91, the subject matter of Examples 79-90 includes, whereinthe means for causing content shared by the content source device to bedisplayed in the sub-display of the display comprises: means forproviding an address of the display to the content source device, andwherein the content source device sends the content to the address.

In Example 92, the subject matter of Examples 79-91 includes, whereinthe means for causing content shared by the content source device to bedisplayed in the sub-display of the display comprises: means forreceiving the content from the content source device; and means forsending the content to the display.

In Example 93, the subject matter of Examples 79-92 includes, whereinthe means for causing content shared by the content source device to bedisplayed in the sub-display of the display comprises: means forreceiving the content from the content source device; and means forrendering the content on the display in the sub-display.

In Example 94, the subject matter of Examples 79-93 includes, whereinthe device is communicatively coupled through a local bus with thedisplay.

In Example 95, the subject matter of Examples 79-94 includes, whereinthe device is communicatively coupled to the display through apacket-based Internet Protocol network.

In Example 96, the subject matter of Examples 79-95 includes, whereinthe display is a touchscreen display and wherein the gesture input is atouch input on the display surface.

In Example 97, the subject matter of Examples 79-96 includes, whereinthe display surface is a projection screen and the display is providedby a projector.

In Example 98, the subject matter of Examples 79-97 includes, means fordisplaying a prompt on the display, the prompt indicating that thedisplay has additional free space not occupied by sub-displays, theprompt offering a user a selection to automatically increase a size ofthe sub-display to fill the additional free space; and means forincreasing the size of the sub-display responsive to receiving anindication to increase the size of the sub-display.

In Example 99, the subject matter of Example 98 includes, wherein theprompt is displayed based upon a determination that the content is beingscaled to fit in the sub-display.

In Example 100, the subject matter of Examples 79-99 includes, whereinthe content is content of a first source and the different content froma second source, and wherein the device further comprises: means fordisplaying a prompt on the display, the prompt suggesting automaticallycreating a new sub-display with content from a third source, the firstand third sources both from the content source device; and means forcreating the new sub-display and displaying the content from the thirdsource responsive to receiving an indication to create the newsub-display.

In Example 101, the subject matter of Example 100 includes, wherein thefirst and third sources are different communication types provided by acommunication application executing on the content source device.

In Example 102, the subject matter of Examples 100-101 includes, whereindisplaying the prompt is responsive to determining that the display hasadditional free space not occupied by sub-displays and that the contentsource device includes the third content source not displayed in asub-display.

In Example 103, the subject matter of Examples 79-102 includes, whereinthe means for identifying the gesture input comprises means forreceiving an indication of the gesture input from the display.

In Example 104, the subject matter of Examples 79-103 includes, whereinthe means for identifying the gesture input comprises means fordetecting the gesture input at a touchscreen of the display.

Example 105 is a computing device for pairing a designated sub-display,the computing device comprising: one or more hardware processors; amemory, storing instructions, which when executed by the one or morehardware processors, causes the one or more hardware processors toperform operations comprising: identifying a gesture input directed to adisplay surface of a display, the gesture input defining a geometry of asub-display on the display, the sub-display encompassing differentcoordinates than a previously specified sub-display; responsive toidentifying the gesture input, creating the sub-display based upon thedefined geometry; displaying a pairing information input area on thedisplay that corresponds to the sub-display; receiving a pairing inputcomprising first pairing information at the pairing information inputarea; linking the sub-display with a content source device based uponmatching the first pairing information from the pairing input withsecond pairing information that is provided by the content sourcedevice; and responsive to linking the sub-display with the contentsource device, causing content shared by the content source device to bedisplayed in the sub-display of the display.

In Example 106, the subject matter of Example 105 includes, wherein thefirst pairing information is first biometric scan data and wherein theoperations of receiving the pairing input comprises causing a biometricscanner to scan a portion of the display corresponding to the pairinginformation input area using an in-display biometric scanner to producethe first biometric scan data.

In Example 107, the subject matter of Example 106 includes, wherein thesecond pairing information is second biometric scan data received with apairing request sent by the content source device, and wherein theoperations of linking the sub-display with the content source devicebased upon matching the first pairing information with the secondpairing information comprises: storing the first biometric scan data ina record of the sub-display; identifying the record of the sub-displayfrom a plurality of other records of other sub-displays based uponmatching the first biometric scan data with the second biometric scandata; updating the record of the sub-display with an address of thecontent source device from the pairing request; and wherein causingcontent shared by the content source device to be displayed in thesub-display of the display comprises: identifying a plurality of packetswith content from the content source device; identifying the sub-displayby matching the address in a header of at least one of the plurality ofpackets with the address of the content source device stored in therecord of the sub-display; and causing the content to be rendered in thesub-display of the display.

In Example 108, the subject matter of Examples 106-107 includes, whereinthe first biometric scan data is one of: fingerprint scan data, humaneye scan data, or facial recognition data.

In Example 109, the subject matter of Examples 105-108 includes, whereinthe pairing information input area comprises a tab visually attached toa border of the sub-display.

In Example 110, the subject matter of Examples 105-109 includes, whereinthe first pairing information is a numeric, alphabetic, or alphanumericcode.

In Example 111, the subject matter of Examples 105-110 includes, whereinthe first pairing information is login credentials of a user account.

In Example 112, the subject matter of Examples 105-111 includes, whereinthe pairing information input area corresponds to the sub-display bylinking the sub-display with the pairing input.

In Example 113, the subject matter of Example 112 includes, wherein thepairing information input area corresponds to the sub-display byvisually linking the pairing information input area with thesub-display.

Example 114 is a method of pairing a designated sub-display, the methodcomprising: using one or more hardware processors: identifying a gestureinput directed to a display surface of a display, the gesture inputdefining a geometry of a sub-display on the display, the sub-displayencompassing different coordinates than a previously specifiedsub-display; responsive to identifying the gesture input, creating thesub-display based upon the defined geometry; displaying a pairinginformation input area on the display that corresponds to thesub-display; receiving a pairing input comprising first pairinginformation at the pairing information input area; linking thesub-display with a content source device based upon matching the firstpairing information from the pairing input with second pairinginformation that is provided by the content source device; andresponsive to linking the sub-display with the content source device,causing content shared by the content source device to be displayed inthe sub-display of the display.

In Example 115, the subject matter of Example 114 includes, wherein thefirst pairing information is first biometric scan data and whereinreceiving the pairing input comprises causing a biometric scanner toscan a portion of the display corresponding to the pairing informationinput area using an in-display biometric scanner to produce the firstbiometric scan data.

In Example 116, the subject matter of Example 115 includes, wherein thesecond pairing information is second biometric scan data received with apairing request sent by the content source device, and wherein linkingthe sub-display with the content source device based upon matching thefirst pairing information with the second pairing information comprises:storing the first biometric scan data in a record of the sub-display;identifying the record of the sub-display from a plurality of otherrecords of other sub-displays based upon matching the first biometricscan data with the second biometric scan data; updating the record ofthe sub-display with an address of the content source device from thepairing request; and wherein causing content shared by the contentsource device to be displayed in the sub-display of the displaycomprises: identifying a plurality of packets with content from thecontent source device; identifying the sub-display by matching theaddress in a header of at least one of the plurality of packets with theaddress of the content source device stored in the record of thesub-display; and causing the content to be rendered in the sub-displayof the display.

In Example 117, the subject matter of Examples 115-116 includes, whereinthe first biometric scan data is one of: fingerprint scan data, humaneye scan data, or facial recognition data.

In Example 118, the subject matter of Examples 114-117 includes, whereinthe pairing information input area comprises a tab visually attached toa border of the sub-display.

In Example 119, the subject matter of Examples 114-118 includes, whereinthe first pairing information is a numeric, alphabetic, or alphanumericcode.

In Example 120, the subject matter of Examples 114-119 includes, whereinthe first pairing information is login credentials of a user account.

In Example 121, the subject matter of Examples 114-120 includes, whereinthe pairing information input area corresponds to the sub-display bylinking the sub-display with the pairing input.

In Example 122, the subject matter of Example 121 includes, wherein thepairing information input area corresponds to the sub-display byvisually linking the pairing information input area with thesub-display.

Example 123 is a machine-readable medium, storing instructions forpairing a designated sub-display, the instructions, when executed by amachine, causes the machine to perform operations comprising:identifying a gesture input directed to a display surface of a display,the gesture input defining a geometry of a sub-display on the display,the sub-display encompassing different coordinates than a previouslyspecified sub-display; responsive to identifying the gesture input,creating the sub-display based upon the defined geometry; displaying apairing information input area on the display that corresponds to thesub-display; receiving a pairing input comprising first pairinginformation at the pairing information input area; linking thesub-display with a content source device based upon matching the firstpairing information from the pairing input with second pairinginformation that is provided by the content source device; andresponsive to linking the sub-display with the content source device,causing content shared by the content source device to be displayed inthe sub-display of the display.

In Example 124, the subject matter of Example 123 includes, wherein thefirst pairing information is first biometric scan data and wherein theoperations of receiving the pairing input comprises causing a biometricscanner to scan a portion of the display corresponding to the pairinginformation input area using an in-display biometric scanner to producethe first biometric scan data.

In Example 125, the subject matter of Example 124 includes, wherein thesecond pairing information is second biometric scan data received with apairing request sent by the content source device, and wherein theoperations of linking the sub-display with the content source devicebased upon matching the first pairing information with the secondpairing information comprises: storing the first biometric scan data ina record of the sub-display; identifying the record of the sub-displayfrom a plurality of other records of other sub-displays based uponmatching the first biometric scan data with the second biometric scandata; updating the record of the sub-display with an address of thecontent source device from the pairing request; and wherein causingcontent shared by the content source device to be displayed in thesub-display of the display comprises: identifying a plurality of packetswith content from the content source device; identifying the sub-displayby matching the address in a header of at least one of the plurality ofpackets with the address of the content source device stored in therecord of the sub-display; and causing the content to be rendered in thesub-display of the display.

In Example 126, the subject matter of Examples 124-125 includes, whereinthe first biometric scan data is one of: fingerprint scan data, humaneye scan data, or facial recognition data.

In Example 127, the subject matter of Examples 123-126 includes, whereinthe pairing information input area comprises a tab visually attached toa border of the sub-display.

In Example 128, the subject matter of Examples 123-127 includes, whereinthe first pairing information is a numeric, alphabetic, or alphanumericcode.

In Example 129, the subject matter of Examples 123-128 includes, whereinthe first pairing information is login credentials of a user account.

In Example 130, the subject matter of Examples 123-129 includes, whereinthe pairing information input area corresponds to the sub-display bylinking the sub-display with the pairing input.

In Example 131, the subject matter of Example 130 includes, wherein thepairing information input area corresponds to the sub-display byvisually linking the pairing information input area with thesub-display.

Example 132 is a device for pairing a designated sub-display, the devicecomprising: means for identifying a gesture input directed to a displaysurface of a display, the gesture input defining a geometry of asub-display on the display, the sub-display encompassing differentcoordinates than a previously specified sub-display; means for,responsive to identifying the gesture input, creating the sub-displaybased upon the defined geometry; means for displaying a pairinginformation input area on the display that corresponds to thesub-display; means for receiving a pairing input comprising firstpairing information at the pairing information input area; means forlinking the sub-display with a content source device based upon matchingthe first pairing information from the pairing input with second pairinginformation that is provided by the content source device; and meansfor, responsive to linking the sub-display with the content sourcedevice, causing content shared by the content source device to bedisplayed in the sub-display of the display.

In Example 133, the subject matter of Example 132 includes, wherein thefirst pairing information is first biometric scan data and wherein themeans for receiving the pairing input comprises causing a biometricscanner to scan a portion of the display corresponding to the pairinginformation input area using an in-display biometric scanner to producethe first biometric scan data.

In Example 134, the subject matter of Example 133 includes, wherein thesecond pairing information is second biometric scan data received with apairing request sent by the content source device, and wherein the meansfor linking the sub-display with the content source device based uponmatching the first pairing information with the second pairinginformation comprises: means for storing the first biometric scan datain a record of the sub-display; means for identifying the record of thesub-display from a plurality of other records of other sub-displaysbased upon matching the first biometric scan data with the secondbiometric scan data; means for updating the record of the sub-displaywith an address of the content source device from the pairing request;and wherein causing content shared by the content source device to bedisplayed in the sub-display of the display comprises: means foridentifying a plurality of packets with content from the content sourcedevice; means for identifying the sub-display by matching the address ina header of at least one of the plurality of packets with the address ofthe content source device stored in the record of the sub-display; andmeans for causing the content to be rendered in the sub-display of thedisplay.

In Example 135, the subject matter of Examples 133-134 includes, whereinthe first biometric scan data is one of: fingerprint scan data, humaneye scan data, or facial recognition data.

In Example 136, the subject matter of Examples 132-135 includes, whereinthe pairing information input area comprises a tab visually attached toa border of the sub-display.

In Example 137, the subject matter of Examples 132-136 includes, whereinthe first pairing information is a numeric, alphabetic, or alphanumericcode.

In Example 138, the subject matter of Examples 132-137 includes, whereinthe first pairing information is login credentials of a user account.

In Example 139, the subject matter of Examples 132-138 includes, whereinthe pairing information input area corresponds to the sub-display bylinking the sub-display with the pairing input.

In Example 140, the subject matter of Example 139 includes, wherein thepairing information input area corresponds to the sub-display byvisually linking the pairing information input area with thesub-display.

Example 141 is a device for designating an input sub-display area, thedevice comprising: one or more hardware processors; a memory, storinginstructions, which when executed by the one or more hardwareprocessors, causes the one or more hardware processors to performoperations comprising: identifying a gesture input directed to a displaysurface of a display, the gesture input defining a geometry of a firstsub-display on the display; responsive to the identifying the gestureinput, creating the first sub-display based upon the geometry; causingcontent from a content source device to be displayed in the firstsub-display of the display; identifying a second gesture input directedto the display surface of the display, the second gesture input defininga geometry of a second sub-display on the display, the secondsub-display having a different location from the first sub-display onthe display; responsive to the second gesture input, linking the secondsub-display to the content source device; identifying an input directedto the second sub-display; and causing the input directed to the secondsub-display to be sent to the content source device.

In Example 142, the subject matter of Example 141 includes, wherein nocontent is displayed on the second sub-display and wherein the inputentered on the second sub-display is not displayed on the secondsub-display.

In Example 143, the subject matter of Examples 141-142 includes, whereinthe input entered on the second sub-display is not displayed on thefirst sub-display.

In Example 144, the subject matter of Examples 141-143 includes, whereinthe operations further comprise: sending the identified input directedto the second sub-display directed to the second sub-display to thecontent source device; and saving the input, by the content sourcedevice, separately from the content from the content source device.

In Example 145, the subject matter of Examples 141-144 includes, whereinthe operations further comprise: sending the identified input directedto the second sub-display to the content source device; and saving theinput, by the content source device, as supplemental data of the contentfrom the content source device correlated to a particular portion of thecontent from the content source device.

In Example 146, the subject matter of Example 145 includes, wherein thecontent from the content source device is a presentation and secondarydata are a plurality of notes, each note of the plurality of notescorrelated to a particular slide of the presentation.

In Example 147, the subject matter of Examples 141-146 includes, whereinthe operations further comprise: identifying a second input directed tothe first sub-display; sending the second input to the content sourcedevice; and causing display of the second input on the firstsub-display.

In Example 148, the subject matter of Examples 141-147 includes, whereinthe operations of linking the second sub-display to the content sourcedevice comprises linking an application executing on the content sourcedevice to the second sub-display, and wherein the operations of causingthe input directed to the second sub-display to be sent to the contentsource device comprises causing the input entered on the secondsub-display to be sent to the application on the content source device.

In Example 149, the subject matter of Example 148 includes, wherein asecond application generates the content displayed on the firstsub-display.

In Example 150, the subject matter of Examples 141-149 includes, whereinthe input is not displayed on the display.

In Example 151, the subject matter of Examples 141-150 includes, whereinthe operations of identifying the input directed to the secondsub-display comprises: receiving an input of a touchscreen of thedisplay at coordinates of the display that are within the secondsub-display, wherein the input is an input in front of the display, agesture touching the display, or an input to an onscreen keyboard on thedisplay.

In Example 152, the subject matter of Examples 141-151 includes, whereinthe operations of linking the second sub-display to the content sourcedevice comprises matching a first sub-display identifier from thedisplay with a second sub-display identifier from the content sourcedevice.

In Example 153, the subject matter of Example 152 includes, wherein theoperations of causing the input directed to the second sub-display to besent to the content source device comprises: determining that the inputcorresponds to the second sub-display; determining that the contentsource device is linked to the second sub-display based upon asub-display information data structure; determining an address of thecontent source device from the sub-display information data structure;and transmitting the input, over a network, to the content source deviceat the address.

In Example 154, the subject matter of Examples 141-153 includes, whereinthe operations further comprise linking the first sub-display to thecontent source device by matching a first sub-display identifier fromthe display with a second sub-display identifier from the content sourcedevice.

Example 155 is a method of designating an input sub-display area, themethod comprising: using one or more hardware processors: identifying agesture input directed to a display surface of a display, the gestureinput defining a geometry of a first sub-display on the display;responsive to the identifying the gesture input, creating the firstsub-display based upon the geometry; causing content from a contentsource device to be displayed in the first sub-display of the display;identifying a second gesture input directed to the display surface ofthe display, the second gesture input defining a geometry of a secondsub-display on the display, the second sub-display having a differentlocation from the first sub-display on the display; responsive to thesecond gesture input, linking the second sub-display to the contentsource device; identifying an input directed to the second sub-display;and causing the input directed to the second sub-display to be sent tothe content source device.

In Example 156, the subject matter of Example 155 includes, wherein nocontent is displayed on the second sub-display and wherein the inputentered on the second sub-display is not displayed on the secondsub-display.

In Example 157, the subject matter of Examples 155-156 includes, whereinthe input entered on the second sub-display is not displayed on thefirst sub-display.

In Example 158, the subject matter of Examples 155-157 includes, sendingthe identified input directed to the second sub-display to the contentsource device; and saving the input, by the content source device,separately from the content from the content source device.

In Example 159, the subject matter of Examples 155-158 includes, sendingthe identified input directed to the second sub-display to the contentsource device; and saving the input, by the content source device, assupplemental data of the content from the content source devicecorrelated to a particular portion of the content from the contentsource device.

In Example 160, the subject matter of Example 159 includes, wherein thecontent from the content source device is a presentation and secondarydata are a plurality of notes, each note of the plurality of notescorrelated to a particular slide of the presentation.

In Example 161, the subject matter of Examples 155-160 includes,identifying a second input directed to the first sub-display; sendingthe second input to the content source device; and causing display ofthe second input on the first sub-display.

In Example 162, the subject matter of Examples 155-161 includes, whereinlinking the second sub-display to the content source device compriseslinking an application executing on the content source device to thesecond sub-display, and wherein causing the input directed to the secondsub-display to be sent to the content source device comprises causingthe input entered on the second sub-display to be sent to theapplication on the content source device.

In Example 163, the subject matter of Example 162 includes, wherein asecond application generates the content displayed on the firstsub-display.

In Example 164, the subject matter of Examples 155-163 includes, whereinthe input is not displayed on the display.

In Example 165, the subject matter of Examples 155-164 includes, whereinidentifying the input directed to the second sub-display comprises:receiving an input of a touchscreen of the display at coordinates of thedisplay that are within the second sub-display, wherein the input is aninput in front of the display, a gesture touching the display, or aninput to an onscreen keyboard on the display.

In Example 166, the subject matter of Examples 155-165 includes, whereinlinking the second sub-display to the content source device comprisesmatching a first sub-display identifier from the display with a secondsub-display identifier from the content source device.

In Example 167, the subject matter of Example 166 includes, whereincausing the input directed to the second sub-display to be sent to thecontent source device comprises: determining that the input correspondsto the second sub-display; determining that the content source device islinked to the second sub-display based upon a sub-display informationdata structure; determining an address of the content source device fromthe sub-display information data structure; and transmitting the input,over a network, to the content source device at the address.

In Example 168, the subject matter of Examples 155-167 includes, whereinthe method further comprises linking the first sub-display to thecontent source device by matching a first sub-display identifier fromthe display with a second sub-display identifier from the content sourcedevice.

Example 169 is a machine-readable medium, storing instructions fordesignating an input sub-display area, the instructions, when executedby a machine, cause the machine to perform operations comprising:identifying a gesture input directed to a display surface of a display,the gesture input defining a geometry of a first sub-display on thedisplay; responsive to the identifying the gesture input, creating thefirst sub-display based upon the geometry; causing content from acontent source device to be displayed in the first sub-display of thedisplay; identifying a second gesture input directed to the displaysurface of the display, the second gesture input defining a geometry ofa second sub-display on the display, the second sub-display having adifferent location from the first sub-display on the display; responsiveto the second gesture input, linking the second sub-display to thecontent source device; identifying an input directed to the secondsub-display; and causing the input directed to the second sub-display tobe sent to the content source device.

In Example 170, the subject matter of Example 169 includes, wherein nocontent is displayed on the second sub-display and wherein the inputentered on the second sub-display is not displayed on the secondsub-display.

In Example 171, the subject matter of Examples 169-170 includes, whereinthe input entered on the second sub-display is not displayed on thefirst sub-display.

In Example 172, the subject matter of Examples 169-171 includes, whereinthe operations further comprise: sending the identified input directedto the second sub-display to the content source device; and saving theinput, by the content source device, separately from the content fromthe content source device.

In Example 173, the subject matter of Examples 169-172 includes, whereinthe operations further comprise: sending the identified input directedto the second sub-display to the content source device; and saving theinput, by the content source device, as supplemental data of the contentfrom the content source device correlated to a particular portion of thecontent from the content source device.

In Example 174, the subject matter of Example 173 includes, wherein thecontent from the content source device is a presentation and secondarydata are a plurality of notes, each note of the plurality of notescorrelated to a particular slide of the presentation.

In Example 175, the subject matter of Examples 169-174 includes, whereinthe operations further comprise: identifying a second input directed tothe first sub-display; sending the second input to the content sourcedevice; and causing display of the second input on the firstsub-display.

In Example 176, the subject matter of Examples 169-175 includes, whereinthe operations of linking the second sub-display to the content sourcedevice comprises linking an application executing on the content sourcedevice to the second sub-display, and wherein the operations of causingthe input directed to the second sub-display to be sent to the contentsource device comprises causing the input entered on the secondsub-display to be sent to the application on the content source device.

In Example 177, the subject matter of Example 176 includes, wherein asecond application generates the content displayed on the firstsub-display.

In Example 178, the subject matter of Examples 169-177 includes, whereinthe input is not displayed on the display.

In Example 179, the subject matter of Examples 169-178 includes, whereinthe operations of identifying the input directed to the secondsub-display comprises: receiving an input of a touchscreen of thedisplay at coordinates of the display that are within the secondsub-display, wherein the input is an input in front of the display, agesture touching the display, or an input to an onscreen keyboard on thedisplay.

In Example 180, the subject matter of Examples 169-179 includes, whereinthe operations of linking the second sub-display to the content sourcedevice comprises matching a first sub-display identifier from thedisplay with a second sub-display identifier from the content sourcedevice.

In Example 181, the subject matter of Example 180 includes, wherein theoperations of causing the input directed to the second sub-display to besent to the content source device comprises: determining that the inputcorresponds to the second sub-display; determining that the contentsource device is linked to the second sub-display based upon asub-display information data structure; determining an address of thecontent source device from the sub-display information data structure;and transmitting the input, over a network, to the content source deviceat the address.

In Example 182, the subject matter of Examples 169-181 includes, whereinthe operations further comprise linking the first sub-display to thecontent source device by matching a first sub-display identifier fromthe display with a second sub-display identifier from the content sourcedevice.

Example 183 is a device for designating an input sub-display area, thedevice comprising: means for identifying a gesture input directed to adisplay surface of a display, the gesture input defining a geometry of afirst sub-display on the display; means for creating the firstsub-display based upon the geometry responsive to the identifying thegesture input; means for causing content from a content source device tobe displayed in the first sub-display of the display; means foridentifying a second gesture input directed to the display surface ofthe display, the second gesture input defining a geometry of a secondsub-display on the display, the second sub-display having a differentlocation from the first sub-display on the display; means for linkingthe second sub-display to the content source device responsive to thesecond gesture input; means for identifying an input directed to thesecond sub-display; and means for causing the input directed to thesecond sub-display to be sent to the content source device.

In Example 184, the subject matter of Example 183 includes, wherein nocontent is displayed on the second sub-display and wherein the inputentered on the second sub-display is not displayed on the secondsub-display.

In Example 185, the subject matter of Examples 183-184 includes, whereinthe input entered on the second sub-display is not displayed on thefirst sub-display.

In Example 186, the subject matter of Examples 183-185 includes, meansfor sending the identified input directed to the second sub-display tothe content source device; and means for saving the input, by thecontent source device, separately from the content from the contentsource device.

In Example 187, the subject matter of Examples 183-186 includes, meansfor sending the identified input directed to the second sub-display tothe content source device; and means for saving the input, by thecontent source device, as supplemental data of the content from thecontent source device correlated to a particular portion of the contentfrom the content source device.

In Example 188, the subject matter of Example 187 includes, wherein thecontent from the content source device is a presentation and secondarydata are a plurality of notes, each note of the plurality of notescorrelated to a particular slide of the presentation.

In Example 189, the subject matter of Examples 183-188 includes, meansfor identifying a second input directed to the first sub-display; meansfor sending the second input to the content source device; and means forcausing display of the second input on the first sub-display.

In Example 190, the subject matter of Examples 183-189 includes, whereinthe means for linking the second sub-display to the content sourcedevice comprises linking an application executing on the content sourcedevice to the second sub-display, and wherein the means for causing theinput directed to the second sub-display to be sent to the contentsource device comprises causing the input entered on the secondsub-display to be sent to the application on the content source device.

In Example 191, the subject matter of Example 190 includes, wherein asecond application generates the content displayed on the firstsub-display.

In Example 192, the subject matter of Examples 183-191 includes, whereinthe input is not displayed on the display.

In Example 193, the subject matter of Examples 183-192 includes, whereinthe means for identifying the input directed to the second sub-displaycomprises: means for receiving an input of a touchscreen of the displayat coordinates of the display that are within the second sub-display,wherein the input is an input in front of the display, a gesturetouching the display, or an input to an onscreen keyboard on thedisplay.

In Example 194, the subject matter of Examples 183-193 includes, whereinthe means for linking the second sub-display to the content sourcedevice comprises means for matching a first sub-display identifier fromthe display with a second sub-display identifier from the content sourcedevice.

In Example 195, the subject matter of Example 194 includes, wherein themeans for causing the input directed to the second sub-display to besent to the content source device comprises: means for determining thatthe input corresponds to the second sub-display; means for determiningthat the content source device is linked to the second sub-display basedupon a sub-display information data structure; means for determining anaddress of the content source device from the sub-display informationdata structure; and means for transmitting the input, over a network, tothe content source device at the address.

In Example 196, the subject matter of Examples 183-195 includes, whereinthe device further comprises means for linking the first sub-display tothe content source device by matching a first sub-display identifierfrom the display with a second sub-display identifier from the contentsource device.

Example 197 is a computing device for designating a sub-display area fora remote content source device, the computing device comprising: one ormore hardware processors; a memory, storing instructions, which whenexecuted by the one or more hardware processors, causes the one or morehardware processors to perform operations comprising: receiving a firstmessage over a network, the first message comprising an identifier of asub-display of a display and an identifier of the remote content sourcedevice, the first message linking the sub-display with the remotecontent source device, the sub-display designated by a gesture inputdirected to a display surface of the display; responsive to receivingthe first message, sending a second message to the remote content sourcedevice over the network, the second message indicating that thesub-display is linked to the remote content source device; receiving,over the network, content from the remote content source device; andcausing display of the content in the sub-display by sending the contentover the network to a display control system, the content containing theidentifier of the sub-display.

In Example 198, the subject matter of Example 197 includes, wherein thefirst message is received over the network from a second computingdevice.

In Example 199, the subject matter of Example 198 includes, wherein theidentifier is entered by a user of the second computing device into anapplication executing on the second computing device and wherein thefirst message is sent by the application.

In Example 200, the subject matter of Examples 198-199 includes, whereinthe first message comprises an address of the display control systemdiscovered by the second computing device based upon receiving abroadcast message from the display control system.

In Example 201, the subject matter of Examples 198-200 includes, whereinthe first message is received as part of a network-based communicationsession with the remote content source device.

In Example 202, the subject matter of Examples 197-201 includes, whereinthe operations further comprise: authenticating the display controlsystem to a network-based communication session; and determining anaddress of the display control system based upon the authenticating.

In Example 203, the subject matter of Examples 197-202 includes, whereinoperations further comprise providing a network-based communicationsession by routing one or more communications between a second computingdevice and the remote content source device to facilitate thenetwork-based communication session.

In Example 204, the subject matter of Examples 197-203 includes, whereinthe first message is received from the display control system.

In Example 205, the subject matter of Examples 197-204 includes, whereinthe operations of causing display of the content comprises sending thecontent to the display control system and wherein the operations furthercomprise: sending the content to a third computing device, the thirdcomputing device a participant of a network-based communication session.

In Example 206, the subject matter of Examples 197-205 includes, whereinthe display control system is a same device as the display.

In Example 207, the subject matter of Examples 197-206 includes, whereinthe identifier of the remote content source device comprises a useridentifier, and wherein the operations further comprise mapping the useridentifier to the remote content source device based upon a user beingauthenticated on the remote content source device.

In Example 208, the subject matter of Examples 197-207 includes, whereinthe operations further comprise: receiving an indication of the gestureinput; and responsive to the indication of the gesture input, creatingthe sub-display by initializing a data structure storing the display.

Example 209 is a method of designating a sub-display area for a remotecontent source device, the method comprising: using one or more hardwareprocessors of a computing device: receiving a first message over anetwork, the first message comprising an identifier of a sub-display ofa display and an identifier of the remote content source device, thefirst message linking the sub-display with the remote content sourcedevice, the sub-display designated by a gesture input directed to adisplay surface of the display; responsive to receiving the firstmessage, sending a second message to the remote content source deviceover the network, the second message indicating that the sub-display islinked to the remote content source device; receiving, over the network,content from the remote content source device; and causing display ofthe content in the sub-display by sending the content over the networkto a display control system, the content containing the identifier ofthe sub-display.

In Example 210, the subject matter of Example 209 includes, wherein thefirst message is received over the network from a second computingdevice.

In Example 211, the subject matter of Example 210 includes, wherein theidentifier is entered by a user of the second computing device into anapplication executing on the second computing device and wherein thefirst message is sent by the application.

In Example 212, the subject matter of Examples 210-211 includes, whereinthe first message comprises an address of the display control systemdiscovered by the second computing device based upon receiving abroadcast message from the display control system.

In Example 213, the subject matter of Examples 210-212 includes, whereinthe first message is received as part of a network-based communicationsession with the remote content source device.

In Example 214, the subject matter of Examples 209-213 includes,authenticating the display control system to a network-basedcommunication session; and determining an address of the display controlsystem based upon the authenticating.

In Example 215, the subject matter of Examples 209-214 includes, whereinthe computing device provides a network-based communication session byrouting one or more communications between a second computing device andthe remote content source device to facilitate the network-basedcommunication session.

In Example 216, the subject matter of Examples 209-215 includes, whereinthe first message is received from the display control system.

In Example 217, the subject matter of Examples 209-216 includes, whereincausing display of the content comprises sending the content to thedisplay control system and wherein the method further comprises: sendingthe content to a third computing device, the third computing device aparticipant of a network-based communication session.

In Example 218, the subject matter of Examples 209-217 includes, whereinthe display control system is a same device as the display.

In Example 219, the subject matter of Examples 209-218 includes, whereinthe identifier of the remote content source device comprises a useridentifier, and wherein the method further comprises mapping the useridentifier to the remote content source device based upon a user beingauthenticated on the remote content source device.

In Example 220, the subject matter of Examples 209-219 includes,receiving an indication of the gesture input; and responsive to theindication of the gesture input, creating the sub-display byinitializing a data structure storing the display.

Example 221 is a machine-readable medium, storing instructions, whichwhen executed by a computing device, causes the computing device toperform operations comprising: receiving a first message over a network,the first message comprising an identifier of a sub-display of a displayand an identifier of a remote content source device, the first messagelinking the sub-display with the remote content source device, thesub-display designated by a gesture input directed to a display surfaceof the display; responsive to receiving the first message, sending asecond message to the remote content source device over the network, thesecond message indicating that the sub-display is linked to the remotecontent source device; receiving, over the network, content from theremote content source device; and causing display of the content in thesub-display by sending the content over the network to a display controlsystem, the content containing the identifier of the sub-display.

In Example 222, the subject matter of Example 221 includes, wherein thefirst message is received over the network from a second computingdevice.

In Example 223, the subject matter of Example 222 includes, wherein theidentifier is entered by a user of the second computing device into anapplication executing on the second computing device and wherein thefirst message is sent by the application.

In Example 224, the subject matter of Examples 222-223 includes, whereinthe first message comprises an address of the display control systemdiscovered by the second computing device based upon receiving abroadcast message from the display control system.

In Example 225, the subject matter of Examples 222-224 includes, whereinthe first message is received as part of a network-based communicationsession with the remote content source device.

In Example 226, the subject matter of Examples 221-225 includes, whereinthe operations further comprise: authenticating the display controlsystem to a network-based communication session; and determining anaddress of the display control system based upon the authenticating.

In Example 227, the subject matter of Examples 221-226 includes, whereinoperations further comprise providing a network-based communicationsession by routing one or more communications between a second computingdevice and the remote content source device to facilitate thenetwork-based communication session.

In Example 228, the subject matter of Examples 221-227 includes, whereinthe first message is received from the display control system.

In Example 229, the subject matter of Examples 221-228 includes, whereinthe operations of causing display of the content comprises sending thecontent to the display control system and wherein the operations furthercomprise: sending the content to a third computing device, the thirdcomputing device a participant of a network-based communication session.

In Example 230, the subject matter of Examples 221-229 includes, whereinthe display control system is a same device as the display.

In Example 231, the subject matter of Examples 221-230 includes, whereinthe identifier of the remote content source device comprises a useridentifier, and wherein the operations further comprise mapping the useridentifier to the remote content source device based upon a user beingauthenticated on the remote content source device.

In Example 232, the subject matter of Examples 221-231 includes, whereinthe operations further comprise: receiving an indication of the gestureinput; and responsive to the indication of the gesture input, creatingthe sub-display by initializing a data structure storing the display.

Example 233 is a computing device for designating a sub-display area fora remote content source device, the computing device comprising: meansfor receiving a first message over a network, the first messagecomprising an identifier of a sub-display of a display and an identifierof the remote content source device, the first message linking thesub-display with the remote content source device, the sub-displaydesignated by a gesture input directed to a display surface of thedisplay; means for, responsive to receiving the first message, sending asecond message to the remote content source device over the network, thesecond message indicating that the sub-display is linked to the remotecontent source device; means for receiving, over the network, contentfrom the remote content source device; and means for causing display ofthe content in the sub-display by sending the content over the networkto a display control system, the content containing the identifier ofthe sub-display.

In Example 234, the subject matter of Example 233 includes, wherein thefirst message is received over the network from a second computingdevice.

In Example 235, the subject matter of Example 234 includes, wherein theidentifier is entered by a user of the second computing device into anapplication executing on the second computing device and wherein thefirst message is sent by the application.

In Example 236, the subject matter of Examples 234-235 includes, whereinthe first message comprises an address of the display control systemdiscovered by the second computing device based upon receiving abroadcast message from the display control system.

In Example 237, the subject matter of Examples 234-236 includes, whereinthe first message is received as part of a network-based communicationsession with the remote content source device.

In Example 238, the subject matter of Examples 233-237 includes, meansfor authenticating the display control system to a network-basedcommunication session; and means for determining an address of thedisplay control system based upon the authenticating.

In Example 239, the subject matter of Examples 233-238 includes, whereinthe computing device provides a network-based communication session byrouting one or more communications between a second computing device andthe remote content source device to facilitate the network-basedcommunication session.

In Example 240, the subject matter of Examples 233-239 includes, whereinthe first message is received from the display control system.

In Example 241, the subject matter of Examples 233-240 includes, whereinthe means for causing display of the content comprises sending thecontent to the display control system and wherein the computing devicefurther comprises: means for sending the content to a third computingdevice, the third computing device a participant of a network-basedcommunication session.

In Example 242, the subject matter of Examples 233-241 includes, whereinthe display control system is a same device as the display.

In Example 243, the subject matter of Examples 233-242 includes, whereinthe identifier of the remote content source device comprises a useridentifier, and wherein the computing device further comprises means formapping the user identifier to the remote content source device basedupon a user being authenticated on the remote content source device.

In Example 244, the subject matter of Examples 233-243 includes, meansfor receiving an indication of the gesture input; and means for,responsive to the indication of the gesture input, creating thesub-display by initializing a data structure storing the display.

Example 245 is a computing device for causing display of notificationswith sub-displays, the computing device comprising: one or more hardwareprocessors; a memory, storing instructions, which when executed, causesthe one or more hardware processors to perform operations comprising:identifying a gesture input directed to a display surface of a display,the gesture input defining a geometry of a sub-display on the display;responsive to the gesture input, creating the sub-display based upon thegeometry and linking the sub-display with a content source device;causing content sent by the content source device to be displayed in thesub-display of the display; receiving a message from the content sourcedevice with information about a notification on the content sourcedevice that, if displayed in the sub-display would otherwise overlapwith the content; causing the display of the notification in anotification region of the display; and causing the sub-display of thedisplay to display the content sent by the content source device withoutthe notification overlapping the content.

In Example 246, the subject matter of Example 245 includes, wherein thenotification region is adjacent to a border of the sub-display.

In Example 247, the subject matter of Examples 245-246 includes, whereinthe notification region is not rendered prior to receipt of the message.

In Example 248, the subject matter of Examples 245-247 includes, whereinthe notification region is a separate sub-display.

In Example 249, the subject matter of Examples 245-248 includes, whereinthe operations further comprise: providing a user interface withselectable options for showing different notification types on thesub-display; receiving a selection of a first notification type todisplay on the sub-display; and wherein causing the display of thenotification comprises determining that the notification is of the firstnotification type.

In Example 250, the subject matter of Example 249 includes, wherein theuser interface is displayed on the content source device.

In Example 251, the subject matter of Examples 249-250 includes, whereinthe user interface is displayed on the display.

In Example 252, the subject matter of Examples 245-251 includes, whereinthe operations further comprise: identifying an input on thenotification region of the display surface, the input matching apredefined gesture to dismiss the notification; and responsive toidentifying the input on the notification region of the display matchingthe predefined gesture to dismiss the notification, causing thenotification to be dismissed by removing the notification informationfrom the display.

In Example 253, the subject matter of Example 252 includes, wherein theoperations of causing the notification to be dismissed comprises sendinga message to the content source device to dismiss the notification.

In Example 254, the subject matter of Examples 245-253 includes, whereinthe operations further comprise: identifying an input on thenotification region of the display surface, the input matching apredefined gesture to expand the notification; and responsive toidentifying the input on the notification region of the display matchingthe predefined gesture to expand the notification, displaying contentcorresponding to the notification over the content shared by the contentsource device in the sub-display.

In Example 255, the subject matter of Example 254 includes, wherein theoperations of displaying content corresponding to the notification overthe content shared by the content source device in the sub-display ofthe display comprises completely replacing the content shared by thecontent source device in the sub-display with the content correspondingto the notification.

In Example 256, the subject matter of Examples 245-255 includes, whereinthe operations further comprise: identifying an input on thenotification region of the display surface, the input matching apredefined gesture to expand the notification; and responsive toidentifying the input on the notification region of the display matchingthe predefined gesture to expand the notification, displaying contentcorresponding to the notification in an automatically created secondsub-display that is in a different location than the sub-display.

In Example 257, the subject matter of Examples 245-256 includes, whereinthe notification was not generated from an application generating thecontent shared by the content source device.

Example 258 is a method of displaying notifications with sub-displays,the method comprising: using one or more hardware processors:identifying a gesture input directed to a display surface of a display,the gesture input defining a geometry of a sub-display on the display;responsive to the gesture input, creating the sub-display based upon thegeometry and linking the sub-display with a content source device;causing content sent by the content source device to be displayed in thesub-display of the display; receiving a message from the content sourcedevice with information about a notification on the content sourcedevice that, if displayed in the sub-display would otherwise overlapwith the content; causing the display of the notification in anotification region of the display; and causing the sub-display of thedisplay to display the content sent by the content source device withoutthe notification overlapping the content.

In Example 259, the subject matter of Example 258 includes, wherein thenotification region is adjacent to a border of the sub-display.

In Example 260, the subject matter of Examples 258-259 includes, whereinthe notification region is not rendered prior to receipt of the message.

In Example 261, the subject matter of Examples 258-260 includes, whereinthe notification region is a separate sub-display.

In Example 262, the subject matter of Examples 258-261 includes,providing a user interface with selectable options for showing differentnotification types on the sub-display; receiving a selection of a firstnotification type to display on the sub-display; and wherein causing thedisplay of the notification comprises determining that the notificationis of the first notification type.

In Example 263, the subject matter of Example 262 includes, wherein theuser interface is displayed on the content source device.

In Example 264, the subject matter of Examples 262-263 includes, whereinthe user interface is displayed on the display.

In Example 265, the subject matter of Examples 258-264 includes,identifying an input on the notification region of the display surface,the input matching a predefined gesture to dismiss the notification; andresponsive to identifying the input on the notification region of thedisplay matching the predefined gesture to dismiss the notification,causing the notification to be dismissed by removing the notificationinformation from the display.

In Example 266, the subject matter of Example 265 includes, whereincausing the notification to be dismissed comprises sending a message tothe content source device to dismiss the notification.

In Example 267, the subject matter of Examples 258-266 includes,identifying an input on the notification region of the display surface,the input matching a predefined gesture to expand the notification; andresponsive to identifying the input on the notification region of thedisplay matching the predefined gesture to expand the notification,displaying content corresponding to the notification over the contentshared by the content source device in the sub-display.

In Example 268, the subject matter of Example 267 includes, whereindisplaying content corresponding to the notification over the contentshared by the content source device in the sub-display of the displaycomprises completely replacing the content shared by the content sourcedevice in the sub-display with the content corresponding to thenotification.

In Example 269, the subject matter of Examples 258-268 includes,identifying an input on the notification region of the display surface,the input matching a predefined gesture to expand the notification; andresponsive to identifying the input on the notification region of thedisplay matching the predefined gesture to expand the notification,displaying content corresponding to the notification in an automaticallycreated second sub-display that is in a different location than thesub-display.

In Example 270, the subject matter of Examples 258-269 includes, whereinthe notification was not generated from an application generating thecontent shared by the content source device.

Example 271 is a machine-readable medium, storing instructions, whichwhen executed by a machine, causes the machine to perform operationscomprising: identifying a gesture input directed to a display surface ofa display, the gesture input defining a geometry of a sub-display on thedisplay; responsive to the gesture input, creating the sub-display basedupon the geometry and linking the sub-display with a content sourcedevice; causing content sent by the content source device to bedisplayed in the sub-display of the display; receiving a message fromthe content source device with information about a notification on thecontent source device that, if displayed in the sub-display wouldotherwise overlap with the content; causing the display of thenotification in a notification region of the display; and causing thesub-display of the display to display the content sent by the contentsource device without the notification overlapping the content.

In Example 272, the subject matter of Example 271 includes, wherein thenotification region is adjacent to a border of the sub-display.

In Example 273, the subject matter of Examples 271-272 includes, whereinthe notification region is not rendered prior to receipt of the message.

In Example 274, the subject matter of Examples 271-273 includes, whereinthe notification region is a separate sub-display.

In Example 275, the subject matter of Examples 271-274 includes, whereinthe operations further comprise: providing a user interface withselectable options for showing different notification types on thesub-display; receiving a selection of a first notification type todisplay on the sub-display; and wherein causing the display of thenotification comprises determining that the notification is of the firstnotification type.

In Example 276, the subject matter of Example 275 includes, wherein theuser interface is displayed on the content source device.

In Example 277, the subject matter of Examples 275-276 includes, whereinthe user interface is displayed on the display.

In Example 278, the subject matter of Examples 271-277 includes, whereinthe operations further comprise: identifying an input on thenotification region of the display surface, the input matching apredefined gesture to dismiss the notification; and responsive toidentifying the input on the notification region of the display matchingthe predefined gesture to dismiss the notification, causing thenotification to be dismissed by removing the notification informationfrom the display.

In Example 279, the subject matter of Example 278 includes, wherein theoperations of causing the notification to be dismissed comprises sendinga message to the content source device to dismiss the notification.

In Example 280, the subject matter of Examples 271-279 includes, whereinthe operations further comprise: identifying an input on thenotification region of the display surface, the input matching apredefined gesture to expand the notification; and responsive toidentifying the input on the notification region of the display matchingthe predefined gesture to expand the notification, displaying contentcorresponding to the notification over the content shared by the contentsource device in the sub-display.

In Example 281, the subject matter of Example 280 includes, wherein theoperations of displaying content corresponding to the notification overthe content shared by the content source device in the sub-display ofthe display comprises completely replacing the content shared by thecontent source device in the sub-display with the content correspondingto the notification.

In Example 282, the subject matter of Examples 271-281 includes, whereinthe operations further comprise: identifying an input on thenotification region of the display surface, the input matching apredefined gesture to expand the notification; and responsive toidentifying the input on the notification region of the display matchingthe predefined gesture to expand the notification, displaying contentcorresponding to the notification in an automatically created secondsub-display that is in a different location than the sub-display.

In Example 283, the subject matter of Examples 271-282 includes, whereinthe notification was not generated from an application generating thecontent shared by the content source device.

Example 284 is a device for displaying notifications with sub-displays,the device comprising: means for identifying a gesture input directed toa display surface of a display, the gesture input defining a geometry ofa sub-display on the display; means for creating the sub-display basedupon the geometry and linking the sub-display with a content sourcedevice responsive to the gesture input; means for causing content sentby the content source device to be displayed in the sub-display of thedisplay; means for receiving a message from the content source devicewith information about a notification on the content source device that,if displayed in the sub-display would otherwise overlap with thecontent; means for causing the display of the notification in anotification region of the display; and means for causing thesub-display of the display to display the content sent by the contentsource device without the notification overlapping the content.

In Example 285, the subject matter of Example 284 includes, wherein thenotification region is adjacent to a border of the sub-display.

In Example 286, the subject matter of Examples 284-285 includes, whereinthe notification region is not rendered prior to receipt of the message.

In Example 287, the subject matter of Examples 284-286 includes, whereinthe notification region is a separate sub-display.

In Example 288, the subject matter of Examples 284-287 includes, meansfor providing a user interface with selectable options for showingdifferent notification types on the sub-display; means for receiving aselection of a first notification type to display on the sub-display;and wherein causing the display of the notification comprisesdetermining that the notification is of the first notification type.

In Example 289, the subject matter of Example 288 includes, wherein theuser interface is displayed on the content source device.

In Example 290, the subject matter of Examples 288-289 includes, whereinthe user interface is displayed on the display.

In Example 291, the subject matter of Examples 284-290 includes, meansfor identifying an input on the notification region of the displaysurface, the input matching a predefined gesture to dismiss thenotification; and means for, responsive to identifying the input on thenotification region of the display matching the predefined gesture todismiss the notification, causing the notification to be dismissed byremoving the notification information from the display.

In Example 292, the subject matter of Example 291 includes, whereincausing the notification to be dismissed comprises sending a message tothe content source device to dismiss the notification.

In Example 293, the subject matter of Examples 284-292 includes, meansfor identifying an input on the notification region of the displaysurface, the input matching a predefined gesture to expand thenotification; and means for, responsive to identifying the input on thenotification region of the display matching the predefined gesture toexpand the notification, displaying content corresponding to thenotification over the content shared by the content source device in thesub-display.

In Example 294, the subject matter of Example 293 includes, wherein themeans for displaying content corresponding to the notification over thecontent shared by the content source device in the sub-display of thedisplay comprises means for completely replacing the content shared bythe content source device in the sub-display with the contentcorresponding to the notification.

In Example 295, the subject matter of Examples 284-294 includes, meansfor identifying an input on the notification region of the displaysurface, the input matching a predefined gesture to expand thenotification; and means for, responsive to identifying the input on thenotification region of the display matching the predefined gesture toexpand the notification, displaying content corresponding to thenotification in an automatically created second sub-display that is in adifferent location than the sub-display.

In Example 296, the subject matter of Examples 284-295 includes, whereinthe notification was not generated from an application generating thecontent shared by the content source device.

Example 297 is at least one machine-readable medium includinginstructions that, when executed by processing circuitry, cause theprocessing circuitry to perform operations to implement of any ofExamples 1-296.

Example 298 is an apparatus comprising means to implement of any ofExamples 1-296.

Example 299 is a system to implement of any of Examples 1-296.

Example 300 is a method to implement of any of Examples 1-296.

1. A computing device for designating a sub-display area for a remotecontent source device, the computing device comprising: one or morehardware processors; a memory, storing instructions, which when executedby the one or more hardware processors, causes the one or more hardwareprocessors to perform operations comprising: receiving a first messageover a network from a second computing device, the first messagecomprising an identifier of a sub-display of a display and an identifierof the remote content source device, the first message linking thesub-display with the remote content source device, the sub-displaydefined by a gesture input directed to a display surface of the display,the remote content source device and the second computing device beingdifferent devices; responsive to receiving the first message, sending asecond message to the remote content source device over the network, thesecond message indicating that the sub-display is linked to the remotecontent source device; receiving, over the network, content from theremote content source device; and causing display of the content in thesub-display by sending the content over the network to a display controlsystem, the content containing the identifier of the sub-display. 2.(canceled)
 3. The computing device of claim 2, wherein the identifier isentered by a user of the second computing device into an applicationexecuting on the second computing device and wherein the first messageis sent by the application.
 4. The computing device of claim 2, whereinthe first message comprises an address of the display control systemdiscovered by the second computing device based upon receiving abroadcast message from the display control system.
 5. The computingdevice of claim 2, wherein the first message is received as part of anetwork-based communication session with the remote content sourcedevice.
 6. The computing device of claim 1, wherein the operationsfurther comprise: authenticating the display control system to anetwork-based communication session; and determining an address of thedisplay control system based upon the authenticating.
 7. The computingdevice of claim 1, wherein operations further comprise providing anetwork-based communication session by routing one or morecommunications between a second computing device and the remote contentsource device to facilitate the network-based communication session. 8.The computing device of claim 1, wherein the first message is receivedfrom the display control system.
 9. The computing device of claim 1,wherein the operations of causing display of the content comprisessending the content to the display control system and wherein theoperations further comprise: sending the content to a third computingdevice, the third computing device a participant of a network-basedcommunication session.
 10. The computing device of claim 1, wherein thedisplay control system is a same device as the display.
 11. Thecomputing device of claim 1, wherein the identifier of the remotecontent source device comprises a user identifier, and wherein theoperations further comprise mapping the user identifier to the remotecontent source device based upon a user being authenticated on theremote content source device.
 12. The computing device of claim 1,wherein the operations further comprise: receiving an indication of thegesture input; and responsive to the indication of the gesture input,creating the sub-display by initializing a data structure storing thedisplay.
 13. A method of designating a sub-display area for a remotecontent source device, the method comprising: using one or more hardwareprocessors of a computing device: receiving a first message over anetwork from a second computing device, the first message comprising anidentifier of a sub-display of a display and an identifier of the remotecontent source device, the first message linking the sub-display withthe remote content source device, the sub-display defined by a gestureinput directed to a display surface of the display, the remote contentsource device and the second computing device being different devices;responsive to receiving the first message, sending a second message tothe remote content source device over the network, the second messageindicating that the sub-display is linked to the remote content sourcedevice; receiving, over the network, content from the remote contentsource device; and causing display of the content in the sub-display bysending the content over the network to a display control system, thecontent containing the identifier of the sub-display.
 14. (canceled) 15.The method of claim 14, wherein the identifier is entered by a user ofthe second computing device into an application executing on the secondcomputing device and wherein the first message is sent by theapplication.
 16. The method of claim 14, wherein the first messagecomprises an address of the display control system discovered by thesecond computing device based upon receiving a broadcast message fromthe display control system.
 17. The method of claim 14, wherein thefirst message is received as part of a network-based communicationsession with the remote content source device.
 18. The method of claim13, further comprising: authenticating the display control system to anetwork-based communication session; and determining an address of thedisplay control system based upon the authenticating.
 19. The method ofclaim 13, wherein the computing device provides a network-basedcommunication session by routing one or more communications between asecond computing device and the remote content source device tofacilitate the network-based communication session.
 20. The method ofclaim 13, wherein the first message is received from the display controlsystem.